# Changelog

The following section will breakdown the additions and updates to the Zadara
Compute Cloud platform (zCompute).

## Version 25.06

Release 25.06 is a major version
that includes new features,
scalability, security and usability improvements, and bug fixes.

For the **main** additions and updates to zCompute, see
[Key Enhancements](key-enhancements.md#key-enhancements).

### Notable fixes

#### Version 25.06.0

* **NK-16363**
  * All API operations on the legacy `/v2/vms` endpoint are deprecated.
  * API operations on the `/v2/compute/vms` are were previously
    case-insensitive.
    In line with other APIs, the API operations on the
    `/v2/compute/vms` endpoint are now case-sensitive only.

## Version 24.03

Release 24.03 is a major version extended by service pack 24.03.1,
that includes new features,
scalability, security and usability improvements, and bug fixes.

It is highly recommended to upgrade to the latest version,
to improve supportability.

* For the purpose of planning an upgrade, see
  [Supported upgrade paths](software-life-cycle.md#lifecycle-supported-upgrade-paths).
* Refer to [Zadara End-Of-Support (EOS) policy](software-life-cycle.md#lifecycle-eos-policy)
  and [Releases support status](software-life-cycle.md#lifecycle-releases-support-status).

For the **main** additions and updates to zCompute, see
[Key Enhancements](key-enhancements.md#key-enhancements).

### Version 24.03.1

#### Online Volume Retype

zCompute 24.03.01 introduces
[Changing a volume’s type (retyping)](http://guides.zadarastorage.com/cs-compute-guide/latest/vm-common-operations.html#changing-a-volume-s-type-retyping).

Users can now change a volume’s **Volume Type** while it remains online.

This feature enables seamless migration between storage classes.
For example, to improve volume performance or adjust their capabilities,
a possible scenario might be retyping HDD-based volumes to SSD.
The system automatically provisions the destination volume
with updated attributes and transfers data without interrupting operations.

#### OS Support and Marketplace Images

The zCompute 24.03.1 Marketplace includes updated and newly added OS and
tool images.

See
[Machine Images Marketplace](https://guides.zadarastorage.com/cs-compute-guide/latest/machine-images.html#machine-images-marketplace),
or go to the **Machine Images > Marketplace** screen in the UI to see
latest available image versions for download.

#### New Volume Types

zCompute 24.03.01 introduces max burst
[Volume Types](http://guides.zadarastorage.com/cs-compute-guide/latest/vm-common-operations.html#volume-types):

* zvtssdmx (SSD)
* zvthddmx (HDD)

#### UI scalability enhancements

Improved performance and rapid responsiveness in page loading have been
implemented in zCompute’s UI web application interface, noticeable
especially when handling large volumes of events and data.

### Version 24.03.0

#### Backup to Object Storage

zCompute 24.03 introduces
[Backup to Object Storage (B2OS)](http://guides.zadarastorage.com/cs-compute-guide/latest/b2os.html).

zCompute Backup to Object Storage (B2OS) extends backup and restore
capabilities beyond local block storage. It enables backing up and
restoring VMs and volumes that are protected by protection-group to and
from Zadara Object Storage systems.

These Zadara Object Storage systems can also reside in different physical
locations than the source zCompute cloud, allowing recovery to any
zCompute cloud in the event of a site-level failure.

On top of this capability, from zCompute v24.03, Protection Groups
also provide VM-level crash-consistent backups. Backup snapshots are taken
as an atomic operation on all volumes of a protected VM, treating the VM’s
volumes as a consistency group.

zCompute B2OS is an integral feature of zCompute.
It provides full backup and restore functionality without requiring
third-party software or installing software agents on protected VMs.

Protection groups are backup policies that define the backup schedule for
protected VMs and volumes, including the backup interval and retention period.

With the introduction of B2OS, you can optionally configure protection groups
to back up protected VMs and volumes to Zadara Object Storage.

Tenant admins can now access the **Management > Configuration** tile’s
**External Endpoints** configuration, required for defining remote Object
Storage.

##### Known limitation

By design, disabling a Protection Group only disables the Protection Group’s
automatic scheduled backups.

A disabled Protection Group still allows the **Trigger Now** action for an
on-the-fly user-initiated backup of the Protection Group’s protected resources.

#### EKS-D Upgrade

The main change that requires this upgrade is the way Kubernetes connections
to EBS volumes.

Another recommended change is to use Kubernetes instances for running the
Kubernetes cluster control nodes, instead of using Auto-Scaling Groups (ASG).
See the
[README](https://github.com/zadarastorage/zadara-examples/tree/main/k8s/eksd#readme)
for EKS-D in the public Zadara Examples GitHub repository.

#### Automatic OS Detection and VM OS Settings

zCompute 24.03 introduces automatic OS detection when creating a VM
instance.

In the **Compute > Instances > Create Instance** dialog, the
**Operating System** field is automatically populated according to the
OS and version detected in the selected source **Image**, **ISO** or
**Volume**. The **Operating System** field is read-only, and cannot be
overwritten by users.

#### zCompute UI Features and Improvements

zCompute 24.03’s new UI features and improvements include
localization for zCompute supported languages:

* English (default)
* German
* Japanese
* Korean
* Portuguese
* Spanish

The **major** UI features and improvements include:

##### Storage Classes Utilization Visualization

MSP Admins can benefit from an improved visualization of utilization per
Storage Class:

* The **Monitoring > Overview** screen:
  * The **Storage Classes Consumption** tile displays a graphic utilization
    summary for each Storage Class.
  * The **Storage Availability & Health** tile displays each Storage Class
    as a color-coded summary status.

    Selecting a Storage Class in this tile redirects to the
    **Storage Management > Storage Classes > [Storage Class]**
    detailed utilization screen.
* The **Storage Management > Storage Classes** screen displays a summary
  table of utilizaion per Storage Class.

  Selecting a Storage Class displays detailed utilization information for
  that Storage Class.

##### AWS Instance Type Aliases

zCompute 24.03 introduces the **Compute > Instance Type Aliases** screen
that displays a table listing each zCompute Instance Type’s AWS Alias,
number of vCPUs and RAM capacity, and number of VM instances per
Instance Type.

##### Placement Rules

From zCompute 24.03, via the **Compute > Rules** screen, MSP Admins can
create, simulate (test VMs for rule compliance), apply and remove rules for
placing VMs with specified VM tags on nodes with specified node tags.

#### V2Z alignments for v24.03

<!-- comment (NK-12198) -->

The V2Z utility in Toolbox Fedora 2.3.5 has several improvements
including:

* Different Windows VirtIO drivers can be used in the V2Z migration process.

  See
  [Replacing Toolbox Windows VirtIO Driver](https://guides.zadarastorage.com/cs-compute-guide/latest/v2z-migration.html#replacing-toolbox-windows-virtio-driver)
  for further details.
  <!-- comment (NK-8324) -->
* The V2Z command line now supports VM and volume names that contain spaces,
  by enclosing the VM or volume name between single quotes.
  <!-- comment (NK-11286) -->

##### Known limitation - V2Z migrates all VMs as UEFI

V2Z migration always migrates VMs to boot in UEFI mode, even if the source
VM is in BIOS mode.

As a workaround to update the migrated VM to boot using BIOS,
after completion of the VM migration run the following `symp` command:

```console
vm update --hw-firmware-type bios <vm-id>
```

<!-- comment (NK-9639) -->

#### Instance Types

Version 24.03 extends the number of VM Instance Types, introducing the Z16
and ZP16 types for memory-intensive use cases.

For more details, see
[Instance Types](https://guides.zadarastorage.com/cs-compute-guide/latest/instance-type.html):

* [Z16 Instance Types](https://guides.zadarastorage.com/cs-compute-guide/latest/instance-type.html#z16-instance-types)
* [ZP16 Instance Types](https://guides.zadarastorage.com/cs-compute-guide/latest/instance-type.html#zp16-instance-types)
* [ZD16 Instance Types](https://guides.zadarastorage.com/cs-compute-guide/latest/instance-type.html#zd16-instance-types)
* [ZD16A Instance Types](https://guides.zadarastorage.com/cs-compute-guide/latest/instance-type.html#zd16a-instance-types)

#### OS Support and new Marketplace Images

The zCompute 24.03 Marketplace includes the upgraded Toolbox Fedora version
2.3.5, along with updated and newly added OS and tool images.

See
[Machine Images Marketplace](https://guides.zadarastorage.com/cs-compute-guide/latest/machine-images.html#machine-images-marketplace),
or go to the **Machine Images > Marketplace** screen in the UI to see
latest available image versions for download.

<!-- comment (NK-12199) -->

#### Direct Subnets in VLANs Management

Known limitation - Direct Subnet in VLAN Management via CLI/API only

On completion of
[Creating a Direct Subnet](https://guides.zadarastorage.com/cs-compute-guide/latest/subnets.html#creating-a-direct-subnet),
the Direct Subnet is immediately available on the specified VPC,
but it is not displayed in the **Account Networking > VLANs Management**
screen.

However, the Direct Subnet’s `network_id` can be viewed using the symp
`vlan-pool vlan get <id>` command. For example:

```console
vlan-pool vlan get 1a8cd9e6-0d7d-4ada-ac5b-0bad3fdd291e

+-----------------------+--------------------------------------+
| id                    | 1a8cd9e6-0d7d-4ada-ac5b-0bad3fdd291e |
| name                  | none                                 |
| account_vlan_pool_id  | 93cb6144-7f28-44c3-9492-0b46d13da88d |
| created_at            | 2024-11-18T11:47:50Z                 |
| guest_network_pool_id | 684703f5-d641-4ddb-be82-62b81a024509 |
| network_id            | cecc1054-538a-467e-a615-83e750ce04b4 |
| project_vlan_pool_id  | adb0d5ba-7927-4813-aea7-d0f0d568d808 |
| updated_at            | 2024-11-18T11:47:51Z                 |
| vlan_tag_id           | 40                                   |
+-----------------------+--------------------------------------+
```

<!-- comment (NK-12200) -->

#### DVS and VLAN Management Granular Policies

From zCompute 24.03, tenant admins can now
provide users with new granular access policies for VLAN management and DVS
network management:

* `DVSReadOnlyAccess`
* `DVSFullAccess`
* `VlanMgmtReadOnlyAccess`
* `VlanMgmtAccountsAssignAccess`
* `VlanMgmtFullAccess`

To apply access policies to users, see
[Managing Users Permissions](https://guides.zadarastorage.com/cs-compute-guide/latest/iam-users.html#managing-users-permissions).

#### Windows KMS with AD / VPC DNS

Windows VMs can now participate in an AD domain, regardless of whether
their license is provided from a central Windows KMS.

#### Large Cluster Systems (LCS) live upgrade

In zCompute versions prior to version 24.03, LCS upgrades involved
coordinating a maintenance window for scheduled downtime.
LCS clusters can now be upgraded live, eliminating the need for downtime,
and without payload disruption.

For more information on LCS, see [Large Cluster Systems (LCS)](#large-cluster-systems-lcs).

<!-- comment

ASG improvements
~~~~~~~~~~~~~~~~

This?
NK-12795
https://zadara.atlassian.net/wiki/spaces/ZEC/pages/2191163393/zCompute+24.03+Internal+QA+Notes#ASG-|-Cluster-after-upgrade-from-23.08-%3E24.03-ASG-does-not-scale-up-(NK-12795)

?? NK-9770 Autoscaling-group state-machine fixes 24.03

?? mostly under the hood for end-users (LCS upgrade related):
NK-9187 ASG | LCS | K8S ASG VMs were restarted during the LCS upgrade -->

### Notable fixes

#### Version 24.03.1

* **NK-16224**

  An issue in the VPC Route Table management method caused
  VPC-to-VPN connectivity problems following a cloud version upgrade.
  The problem has been resolved, restoring normal connectivity behavior.
* **NK-14521**

  There was an issue where some VM snapshots were not displayed on the
  **VM Details** page, under the **Snapshots** tab. This is now resolved.
* **NK-12966**

  A UI problem caused a browser exception error warning page to appear
  when navigating between the **Restore Protection Group** and
  **Protection Group Snapshots** views.
  This is resolved, and navigation flow now works without error.
* **NK-16260**

  An issue was discovered where VM names appeared as UUIDs in Snapshot
  filters. This is now resolved, displaying VM names.
* **NK-15993**

  A UI issue prevented users from resizing columns in the **Events** page.
  This is resolved, so that users can resize all columns.

#### Version 24.03.0

The V2Z utility in Toolbox Fedora 2.3.5 includes the following fixes:

* **NK-11267**

  A V2Z migration case was discovered, where an invalid parameter resulted
  in a late fail.

  This has been resolved with added initial validation checks rendering a
  fast fail when invalid.
* **NK-11575**

  V2Z preflight checks were failing for minimal permission users.

  This has been resolved so that the V2Z preflight checks succeed for
  users with the minimal required permissions.

## Version 23.08

Release 23.08 is a major version extended by service packs 23.08.1, 23.08.2,
23.08.3 and 23.08.4,
that includes new features,
scalability, security and usability improvements, and bug fixes.

### Version 23.08.4

#### Taikun machine image in Marketplace

From version 23.08.4, the new Taikun image is downloadable from zCompute’s
[zCompute Machine Images Marketplace](https://guides.zadarastorage.com/cs-compute-guide/latest/machine-images.html#zcompute-machine-images-marketplace).
This machine image provides an advanced managed Kubernetes service, with
multiple cluster management from a single pane of glass.

For more information, see:

* [Kubernetes Application Delivery with Taikun and Zadara](https://www.zadara.com/about/partners/taikun-cloudworks/)
* Video -
  [Deploying & Managing Kubernetes Clusters & Applications on Zadara Edge Cloud with Taikun CloudWorks](https://taikun.cloud/videos/deploying-managing-kubernetes-clusters-applications-on-zadara-edge-cloud-with-taikun-cloudworks/)
  <iframe class="embed-responsive-item"
           id="youtubeplayer"
           type="text/html"
           width="640"
           height="390"
           style="border: none"
           src="//www.youtube.com/embed/DrC7rwFEgnw"
           allowfullscreen="allowfullscreen">
  </iframe>

For more information on Machine Images and the zCompute Machine Images
Marketplace, see
[zCompute Machine Images Marketplace](https://guides.zadarastorage.com/cs-compute-guide/latest/machine-images.html#zcompute-machine-images-marketplace)
in the Compute Guide.

#### VPC Peering for Zadara Edge Clouds

From version 23.08.4, zCompute provides a pfSense machine image in the
[zCompute Machine Images Marketplace](https://guides.zadarastorage.com/cs-compute-guide/latest/machine-images.html#zcompute-machine-images-marketplace)
that enables routing network traffic between VPCs in separate zCompute
clouds, using private IPv4 addresses.

This implementation uses IPSec technology for securing private connections
between instances communicating with each other in a Zadara Edge Cloud.

For more information see
[VPC Peering for multiple Zadara Edge Clouds](https://guides.zadarastorage.com/cs-compute-guide/latest/vpc-peering-edge-cloud.html)
in the zCompute User Guide.

#### VPN Service for Zadara Edge Clouds

From version 23.08.4, zCompute provides a pfSense machine image in the
[zCompute Machine Images Marketplace](https://guides.zadarastorage.com/cs-compute-guide/latest/machine-images.html#zcompute-machine-images-marketplace)
that enables remote access to a VPC.

The implementation uses OpenVPN technology for securing private connections
between remote clients and Zadara Edge Cloud resources.

For more information see
[VPN Service for Zadara Edge Clouds](https://guides.zadarastorage.com/cs-compute-guide/latest/vpn-edge-cloud.html)
in the zCompute User Guide.

### Version 23.08.1

#### Language support in zCompute UI

From version 23.08.1, the zCompute UI supports the following languages:

* English (default)
* German
* Japanese
* Korean
* Portuguese
* Spanish

For more information, see
[zCompute UI Language support](https://guides.zadara.com/cs-compute-guide/latest/index.html#zcompute-ui-language-support)
in the Compute Guide.

#### Oracle Linux OS support

zCompute 23.08.1 has been extended to support the following OS types
and versions for VM images:

| Type   | Distribution   |   Version |
|--------|----------------|-----------|
| Linux  | Oracle         |         7 |
| Linux  | Oracle         |         8 |
| Linux  | Oracle         |         9 |

For other supported OS types and versions for VM images, see
[New OS types and versions](#enhancements-os-types).

#### ECDSA key support

From version 23.08.1, zCompute supports use of ECDSA keys for VMs.
It is possible to import and successfully use a public key in
SSH or RSA (PEM) formats.

##### Known issues and limitations

* For Windows password recovery (**Get Windows Password** dialog), the
  UI only accepts keys in RSA (PEM) format.
* In version 23.08.0 and earlier, zCompute does not accept keys for VMs in
  SSH formats, but only in RSA (PEM) format.

#### Toolbox Fedora 2.3.1

The new Toolbox Fedora 2.3.1 VM is available in the zCompute UI’s
**Machine Images > Marketplace**.
This is a Fedora VM that includes the
option to migrate VMware VMs, using the V2Z migration utility.
See [Migrating VMware VMs to zCompute](https://guides.zadara.com/cs-compute-guide/latest/v2z-vmware-migration.html)
in the Compute Guide.

#### Tag support in Subnet creation

From version 23.08.1, zCompute supports the addition of tags in a Subnet
creation operation.
See [Creating a Subnet](https://guides.zadarastorage.com/cs-compute-guide/latest/subnets.html#creating-a-subnet)
in the zCompute User Guide.

This is useful mainly with Terraform and auto-deployments
with EKS-D.

### Version 23.08.0

#### Packaged zCompute EKS-D image

EKS-D is Amazon’s open-source implementation of its Elastic Kubernetes Service
(EKS).

zCompute 23.08 provides customers with the ability to deploy and use the
prepackaged Kubernetes solution in zCompute clusters.

The zCompute EKS-D image is downloadable in the zCompute UI at
**Machine Images > Marketplace**.

The zCompute 23.08 EKS-D solution is verified, and certified by Kasten
(acquired by Veeam). Thus, zCompute 23.08 can also provide a
backup, recovery and migration solution for Kubernetes over zCompute.

For more information and examples, see the
[README](https://github.com/zadarastorage/zadara-examples/blob/main/k8s/eksd/README.md)
in Zadara’s examples repository in GitHub.

#### Distributed Virtual Switch (DVS)

Distributed Virtual Switch (DVS), is a zCompute networking model which
provides layer 2, VLAN-based networking functionality for VMs running on
zCompute.

The idea behind DVS is to provide customers who are more familiar with
legacy virtualization platforms (for example, VMware, Hyper-V, Nutanix, etc.) with a
networking environment similar to the simplified networking model found in
such legacy environments.

DVS provides simple OSI layer 2 VLAN-based switched networking functionality
for VMs.
DVS networks can be used to interconnect VMs attached to them, as well as
for connecting these VMs with other network entities that reside in the
hosting data center (servers, routers, storage devices, etc.).

The DVS model provides physical switching L2, VLAN-based connectivity,
whereas VPC provides a rich IP networking platform (route-tables, subnets,
security groups, Internet gateways, DNS, Elastic IPs, etc.) alongside other
cloud-native services that are independent of physical switching configuration,
such as auto-scaling groups, load-balancers, etc.

Zadara zCompute supports both VPC and DVS networking modes in coexistence:
A single zCompute account (tenant), can have multiple VPC-based projects
alongside DVS-based projects.

DVS was introduced in zCompute 22.09.
In version 23.08, the DVS infrastructure was enhanced with improved
management of VLANs for the purpose of creating guest networks.
Cloud admins allocate VLANS to an account’s VLAN pool.
Tenant admins can allocate VLANs from the account’s VLAN pool to a DVS project
pool.
`Member`-role users can now create VLAN-type guest networks by allocating
VLANs from a DVS project’s VLAN pool.

For more information about DVS, see
[Distributed Virtual Switch (DVS)](https://guides.zadarastorage.com/cs-compute-guide/latest/networking-dvs.html)
in the zCompute User Guide.

<a id="migrating-to-a-dvs-project"></a>

##### Migrating to a DVS project

VMware VMs that are migrated to zCompute using the V2Z migration process
result in VMs in a rich VPC networking type project, rather than in the
simpler DVS networking alternative.

Tenants who prefer the simpler DVS networking model can further migrate
their newly migrated zCompute VPC VMs to a DVS project.

To migrate VMs to a DVS project, see [Migrating a VM to a DVS project](https://guides.zadarastorage.com/cs-compute-guide/latest/v2z-vmware-migration.html#migrating-to-a-dvs-project)
in the Compute Guide.

<a id="volume-types"></a>

#### Volume types

From version 23.08, zCompute simplifies storage management with volume types
that provide users with a range of options to meet their storage requirements,
balancing factors such as performance, cost, and specific workload demands.

By abstracting the underlying storage infrastructure, volume types simplify
storage management and allow users to focus on selecting the appropriate type
for their zCompute needs.

Different volume types offer varying levels of input/output operations per
second (IOPS) and throughput, which determine the storage performance.

#### Auto-scaling group tags propagation to VMs

For easier classification of VMs, tags can be created and applied to
auto-scaling groups. Based on the auto-scaling group configuration,
tags can be automatically propagated to VM instances during their launch.

For more information, see
[Auto-Scaling Groups](https://guides.zadarastorage.com/cs-compute-guide/latest/auto-scaling-groups.html)
in the Compute Guide.

<a id="id13"></a>

#### VM Instance Types

VM instance types are templates defining the resources used by a VM instance.
A VM’s instance type is determined by selecting one of the permitted types,
during the creation flow of the VM instance.

zCompute 23.08 has been enhanced with support for a large number of new
Zadara instance types.

For more information, see
[Instance Types](https://guides.zadarastorage.com/cs-compute-guide/latest/instance-type.html)
in the Compute Guide.

#### VM network interface, Route Table, and Network Reset

zCompute 23.08 introduces soft and hard reset support for VM network interfaces,
route-tables and networks.

* A **soft reset** rebinds all ports, and is available to admins, tenant
  admins and members.
* A **hard reset** resets the service, and is limited to admins and
  tenant admins.

In the zCompute User Guide, see:

* [Subnet Operations](https://guides.zadarastorage.com/cs-compute-guide/latest/subnets.html#subnet-operations)
* [Route Table Operations](https://guides.zadarastorage.com/cs-compute-guide/latest/route-tables.html#route-table-operations)
* [Network Interfaces](https://guides.zadarastorage.com/cs-compute-guide/latest/network-interfaces.html#network-interface-operations)

#### VPC DNS status reporting improvements

In zCompute 23.08, the UI is enhanced with detailed VPC DNS status
information, with key benefits such as:

* Reducing time to resolve VPC DNS issues
* Logging events of CoreDNS VM status and DNS health status changes

When DNS is enabled for a VPC, its DNS VM status and DNS health status are
checked once per minute and reported for the VPC.

See
[VPC DNS status](https://guides.zadarastorage.com/cs-compute-guide/latest/networking-vpc.html#view-vpc-dns-status)
in the zCompute User Guide.

#### Instance Profile

An instance profile is a container for an AWS IAM role. It can be used to
pass role information to an EC2 instance when the instance starts. When
an AWS IAM role, embedded in an instance profile, is attached to an
instance, its credentials become permanent.

In zCompute 23.08, the UI has been enhanced with support for
[Instance Profile](https://guides.zadarastorage.com/cs-compute-guide/latest/aws-roles.html#instance-profiles)
management (zCompute User Guide),
and implementing instance profiles in the **Config** tab of
[Creating VM Instances](https://guides.zadarastorage.com/cs-compute-guide/latest/vm-operations.html#creating-vm-instances)
(Compute Guide).

#### API Audit Trail for Admin/Tenant-Admin

zCompute 23.08 has been enhanced by a security auditing feature, and
provides API capability for integration with 3rd-party solutions.
Admins and tenant admins can query API calls logging POST, PUT, PATCH and
DELETE calls for a 60-day retention period.

See
[api-trail](https://guides.zadara.com/cs-compute-guide/latest/symp-ref-guide/api-trail.html)
in the Symp CLI reference.

#### VPC Endpoint

From zCompute 23.08, a VPC endpoint allows a VM to access the system APIs
using an internal VPC IP address.

The API is available on the internal IP addresses 169.254.64.2 and
169.254.64.3, assuming that the cluster has more than 2 nodes.
The system also provides a DNS record resolvable to these IPs when the VPC
DNS service (coredns) is up, and a valid certificate is installed.

The DNS name:

* When a valid certificate with a wildcard CN is used, the DNS name is:
  `cloud.<parent-domain>`.

  For example, the DNS name for
  CN `*.example.com` is `cloud.example.com`.
* When a valid certificate with a regular CN is used, the DNS name is the
  same as the CN.

  For example, the DNS name for
  CN `test.example.com` is also `test.example.com`.

The cluster’s DNS name is returned in the `cluster_url` property of the
VM’s metadata response. For example, for a cluster with a certificate with
the CN `*.example.com`:

```console
curl -s http://169.254.169.254/openstack/latest/meta_data.json | jq -c .cluster_url

"https://cloud.example.com"
```

#### GRE traffic over EIPs

From zCompute 23.08, Generic Routing Encapsulation (GRE) traffic over
Elastic IP addresses (EIPs) is supported.

GRE Tunneling is a tunneling protocol that encapsulates network
layer protocols inside virtual point-to-point links over an Internet
Protocol network. The tunnel source and tunnel destination addresses on
each side identify the two endpoints.
GRE packets travel directly between the two endpoints through a virtual
tunnel.

##### GRE via NAT gateways - known limitation

GRE via Network Address Translation (NAT) gateways is not supported.

#### VM BIOS auto-reboot

Until zCompute 23.08, there was an issue of Windows VMs getting stuck after
BSOD and reboot, due to no boot device available at the time of the restart.

zCompute 23.08 delivers an enhanced configuration for the VM’s BIOS to keep
auto-rebooting the VM if no boot device is found after 10 seconds.

<a id="enhancements-os-types"></a>

#### New OS types and versions

zCompute 23.08 has been extended to support the following OS types
and versions for VM images:

| Type    | Distribution   | Version   |
|---------|----------------|-----------|
| Linux   | Alma           | 9         |
| Linux   | Alma           | Other     |
| Linux   | Rocky          | 9         |
| Linux   | Rocky          | Other     |
| Linux   | Debian         | 11        |
| Linux   | Fedora         | 34        |
| Linux   | Fedora         | 35        |
| Linux   | Fedora         | 36        |
| Linux   | Fedora         | 37        |
| Linux   | RHEL           | 9         |
| Linux   | Ubuntu         | 22.04     |
| Windows | Windows Server | 2022      |

#### V2Z utility improvements

The V2Z utility supports easy migration from supported
virtualization platforms to zCompute projects.

In zCompute 23.08, the V2Z utility has been extended to support
[VM Instance Types](#id13) and [Volume types](#volume-types) instead of
storage pools.

##### V2Z custom instance types - known limitation

In the V2Z utility, custom instance types are not allowed for most users.

#### Security

In zCompute 23.08, an updated zCompute Toolbox Fedora image (version 2.3.1)
has been released with enhanced security hardening. The Toolbox image is
downloadable in the zCompute UI at **Machine Images > Marketplace**.

### Notable fixes

#### Version 23.08.4

* **NK-10748**:

  A rare edge case was discovered that could theoretically cause a data
  corruption.

  This was resolved despite the unlikely probability of it occurring in any
  current production implementation.

#### Version 23.08.3

* **NK-9822**:

  Fixes related to ASG behavior:
  * Cases were reported where scaling up or down stopped after an upgrade.
  * VMs rebooted during an LCS upgrade.

  These issues are resolved.
* **NK-9297**:

  VSC migration failed due to an error calculating reserved capacity limits.
  This is resolved.
* **NK-9638**:

  The api-trail was accumulating high volume of events before clearing them.
  This is resolved, so that garbage collection is triggered according to
  event count or time interval, whichever is reached first.
* **NK-9967** and **NK-10039**:

  Japanese and Korean localizations in the UI are updated with corrections.

#### Version 23.08.1

* **NK-8656**:

  For tenants that have LDAP configured, the only local zCompute user account
  that should be usable is the tenant admin account (for break-the-glass
  scenarios.)
  In a rare case that was discovered, when the LDAP disconnected, the local
  tenant admin was unable to sign on. This is now resolved.

## Version 22.09

Release 22.09 is a major version extended by service packs 22.09.1, 22.09.2,
22.09.3 and 22.09.4, that include new features,
scalability, security and usability improvements, and bug fixes.

The following section will breakdown the **main** additions and updates to the
Zadara Compute platform (zCompute).

### AWS ELB-compatible APIs in zCompute

Previously, zCompute’s LBaaS supported one certificate per listener.

From release 22.09.4, zCompute supports multiple TLS / SSL certificates for
a single listener on the same TCP port servicing HTTPS.
This feature is currently supported in Symp and AWS API.
It will be introduced into the zCompute UI at a later date.

<a id="large-cluster-systems-lcs"></a>

### Large Cluster Systems (LCS)

With this version of Zadara zCompute, Zadara takes a leap into new use
cases that require much larger clusters at the Edge.
To support larger applications, especially data analytics, a new deployment
topology was added that allows for 75 physical nodes and upwards
(exact numbers to be published in the future).
Until now, the maximum official cluster size was 16 nodes, with engineering
exceptions for specific customers to extend to 24 nodes.
This is referred to as the standard configuration.

Moving to an LCS cluster is required when there’s a need to grow a cluster
beyond 16 nodes.
An LCS cluster can start with one compute node plus three control-only
nodes, but this minimal implementation is not cost-effective for general
business use.
Control nodes can’t be used to run instances.
This has an impact on the floor, but not on the instance pricing.
Zadara’s solution architect will recommend whether to use the standard or
LCS configuration per system, based on customer needs.

### Distributed Virtual Switch (DVS) mode

Release 22.09 introduces a new simplified network model which gives our
customers the ability to place virtual machines (instances) directly on a
VLAN-based mode which extend to the data center switches.
This is similar to other common data center virtualization solutions,
(such as VMWare, HyperV, etc.).

Essentially, tenants can regard DVS mode projects as a single virtual
switch with uplinks to the data center switches.
Tenants see networks that are set up by the cloud admin,
and when choosing to create a network interface for the VM on the network,
the system automatically adds them to a VLAN-tagged port on the virtual
switch.

A network in a DVS mode project represents a VLAN-tagged L2 network.
The VLAN tag itself is hidden from the users.
It’s the admin’s choice whether to disclose the VLAN tag ID to users via
the network name or description.
VMs will get a network interface with a MAC address, but without an IP.
Tenants may assign an IP address manually or via a DHCP server
(connected to their physical network).

Differences between VPC mode and DVS mode projects:

![vpc-vs-dvs](media/vpc-vs-dvs.png)

#### Cloud admin capabilities

Admins can create a project in DVS mode.

Admins can create a VLAN-based network providing the VLAN tags.
Note that the VLAN tags provided must match the physical switches
configuration in the data centers (tagged ports, etc.).

When creating a zCompute account for tenants, the default project is a VPC
project.
Tenants that require a DVS project must open a ticket to Zadara Support or
to their MSP system admin to create a DVS project and the desired networks.

#### DVS project overview

DVS projects have limited features and options in the tenant UI:

![dvs-user-options](media/dvs-user-options.png)

#### DVS project and network creation

DVS project and network creation are restricted to admins only, and are
included here solely for informative and completeness purposes.

* DVS project creation
  ![create-dvs-project](media/create-dvs-project.png)
  ![projects-dvs-project](media/projects-dvs-project.png)
* DVS network creation
  ![create-dvs-network](media/create-dvs-network.png)

#### DVS - known issues

* **NK-3898**: On DVS projects, users are not able to set project level
  tagging that later propagates to virtual machines.
  Workaround: Once the VM has been created, users can navigate to the VM
  and add the tag.
* **NK-4598**: The DVS instance tab shows the key pair option while
  restoring the snapshot backup, even though key pairs are not supported in
  this version.
* **NK-4606**: The DVS networks overview displays a warning message to
  clear the filters, even though no filter is applied. Click `Clear Filters`
  to display the overview.

### Premium instance types

Release 22.09.2 introduces premium instance types in heterogeneous hardware
clusters, based on the 3rd-Gen Intel Xeon Platinum 8362 processor.
This is applicable to both NVMe and non-NVMe instances.
Please contact sales for more information.

### Direct subnet external router

A direct subnet is an IP subnet in a VPC that spans out of a zCompute
system over a VLAN to the datacenter network.
Direct subnets allow communication between physical-network and VPC
entities, for example, VMs within a VPC connected to a VPSA.

In many cases, the tenant may provide a data center router for VMs in the
VPC, including VMs attached to the direct subnet, to access data center
subnets that are not on the direct subnet.

Until now, there was a workaround that required manually creating a network
interface to represent the external router, so the user can set the proper
routes in the VPC route tables.
Now the administrator can add the IP of the external router during the
creation of the direct subnet, and the system will be able to route to this
router.

![create-direct-subnet](media/create-direct-subnet.png)

![view-direct-subnet](media/view-direct-subnet.png)

The system now creates the ENI that can be used for routing rules, making
it easy to add a route towards an external network using ENI.

![create-route](media/create-route.png)

### Auto-Scaling Groups: Added missing compatibility with AWS API

This version supports the Cluster Autoscaler in Kubernetes setup through
the zCompute Auto-Scaling Groups. Cluster Autoscaler is a tool that
automatically adjusts the size of the Kubernetes cluster when one of the
following conditions is true:

* There are pods that failed to run in the cluster due to insufficient
  resources. In this case, zCompute ASG provisions a new node
  (zCompute VM instance), and schedules the failed pods to run on it.
* There are nodes (zCompute VMs) in the cluster that have been underutilized
  for an extended period of time, and their pods can be placed on other
  existing nodes (zCompute VMs).

Cluster Autoscaler utilizes zCompute Instance Auto-Scaling Groups to manage
node groups. Cluster Autoscaler typically runs as a deployment in your
cluster.

![asg-cluster-autoscaler](media/asg-cluster-autoscaler.png)

<!-- comment - placeholder

Kubernetes Cluster Autoscaler preparation steps
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

<placeholder for steps> -->
<!-- comment - hide until KMS proxy feature is finalized

Service proxy for Microsoft KMS - managing Windows licensing
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

zCompute now supports a proxy for Windows virtual machines on any VPC subnets,
private or public, to access a Key Management Service (KMS) server.
When Zadara provides the license keys, Zadara Operations sets up, configures
and maintains this KMS.
When the MSP or the private cloud administrator provide the licenses, the
cloud operator sets up, configures and maintains the KMS.
Until now, Windows VMs either required network access to the KMS, or the
license to be provided manually with activation keys.

Windows licensing proxy is designed to support the Windows SPLA license
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* KMS is an activation service that allows organizations to activate
  systems within their own network.
* Eliminates the need for individual computers to connect to Microsoft for
  product activation.
* The KMS server can be easily configured. It can be co-hosted on a system
  that provides other services.

Prerequisites for MSP and system admins who want to use their own licenses
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* KMS Windows server providing the license key.
* Recommended: Windows images that automatically consume licenses via this
  mechanism. Documentation and instruction will be provided on how to use
  the customer's own image to retrieve the activation keys, either by
  cloud-init or direct access to the KMS server.

Please contact Zadara Support for more information.

Zadara-provided licenses
^^^^^^^^^^^^^^^^^^^^^^^^

Although the zCompute system now has the mechanism, Zadara still needs to
build a global license server to support provisioning licensing for all
Zadara-managed clouds. This is expected to take place in the upcoming few
months. Once completed, Zadara will also provide preconfigured Windows
images to use this mechanism.

Service proxy for Microsoft KMS managing Windows licensing - known issues
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* zCompute:

  * **NK-3910**: Windows Client VM can't activate the license when the VPC
    Coredns has a private IP.

  * DVS mode limitations:

    This proxy is not available in DVS mode.
    To use a KMS server on DVS mode, the Windows VM must have a route to
    the KMS server. It is the MSP admin's responsibility to provide the VM
    with this route and the location of the KMS server.

* Microsoft:

  You can activate Microsoft volume products using the KMS server if the
  following required minimum number of KMS clients (activation threshold)
  are met:

  * Windows Desktop OSs: 25
  * Windows Server OSs: 5 -->
<!-- comment - Is it possible to license Windows Desktop OSes? -->

### LDAP self-service enhancement

This version supports LDAP self-service.
Tenants who wish to use their LDAP to login to their account can now set up
their own LDAP connection without opening a ticket to Zadara Support to do
it for them.

As part of the self-service, the account now supports a “local”/”Break glass”
tenant-admin that can configure and enable the LDAP connection.

* All other users should come from LDAP.
* LDAP users can’t add more local users or groups. Only the local tenant
  admin user is allowed to do so.

#### How to connect LDAP to your tenant account

1. Log into your account as tenant admin.
2. Click the  “Identity Provider” and follow the instructions to fill in
   the details.

zCompute also enhanced the user lists. Now, the lists are split into tabs:

* **Users**: list of local users.
* **LDAP Users**: list of LDAP users.

![ldapTab](media/ldapTab.png)

### Protect admin password change

Admin users won’t be able to set a new password for themselves using the
`user update` API. This is to prevent someone from hijacking an open
session and changing the password without knowing the old password.
If admins want to change their own password, they can use the
`user change-password` API, that requires them to provide the old
password in order to proceed with the change.

### Disabling the VPC DNS allows direct connection to the tenant DNS

In a Microsoft Active Directory (AD) environment, the AD maintains the DNS
and VMs cannot use the VPC DNS. With this feature, tenants can provide
the AD DNS to the Windows VMs via the DHCP protocol.

When using zCompute in VPC mode with a VPC DNS service enabled, the
instances are configured to access the internal VPC DNS which serves as a
caching DNS for external resolvers, in addition to adding the VPC entries.
In this mode, VMs can’t be auto-registered to the external DNS.
This specifically impacts Windows AD environments, where virtual machines
are registered to the domain DNS while joining the domain.

Until now, disabling the VPC DNS only disabled the VPC level DNS, but still
provided a subnet level DNS. The only solution for Windows domain DNS was
to statically configure the Windows OS to point to the domain DNS instead
of the value provided by zCompute DHCP server.

Now, when disabling the VPC DNS, the name server entries added to the DHCP
options will be passed directly to the VMs, so they can stop using the
workaround.

**Steps**:

1. Create a DHCP option, and configure a DNS domain with an external address.

   ![create-dhcp-options-details](media/create-dhcp-options-details.png)
   ![create-dhcp-options-servers](media/create-dhcp-options-servers.png)
2. Disable DNS in the VPC, by unchecking the check box.

   ![modify-vpc-dns-disabled](media/modify-vpc-dns-disabled.png)
3. Attach this DNS option to the VPC.

   ![attach-dhcp-options](media/attach-dhcp-options.png)

### UI now warns about volumes that are not connected to any VM

As with many other cloud services, there are many scenarios when deleting a VM
that data disks are not deleted with the VM, since volumes are entities that
at any given time may be detached from one instance and connected to another.
Volumes can also be detached from the VM and remain in the account as long as
needed.

In some cases, customers forget to delete volumes after deleting the VMs,
and then are billed at the end of the month.
To avoid surprises, the system now informs when there are unattached volumes,
so that customers may go and delete them if they were not intentionally kept.

Possible scenarios where volumes remain unattached:

* Using the APIs, and forgetting to delete the data volumes after deletion
  of the VMs.
* Using Terraform, and Terraform encountered a failure during the teardown
  process.
* Using the GUI to delete a VM or multiple VMs, and forgetting to delete
  their data volumes.

![volumes-non-attached-storage](media/volumes-non-attached-storage.png)
![volumes-non-attached-instances](media/volumes-non-attached-instances.png)

#### Volumes that are not connected to any VM - known issue

* **NK-4242**: ‘Default volumes’ are impossible to delete.
  We are experiencing difficulties in deleting what appear to be
  “non attached volumes”.
  Although the UI does not show the attachment, the volumes are still
  attached to the host.

### Enhanced remote DNS status validation

Until now, the validation only checked that the name server provided to the
VPC DNS is responding to queries.
Now there is a validation that the DNS can resolve a known destination
(default is google.com).

If the provided DNS server can’t resolve this well-known address, the DNS is
reported as degraded.

#### Enhanced remote DNS status validation - known limitations

* The DNS status for VPCs without a valid route to an IGW will always be
  “Degraded” when the default DHCP option with the google DNS is used.
* If tenants set their own name server in the VPC’s DHCP options,
  and their name server is unreachable via the VPC gateway or it can’t
  resolve google.com, the VPC state will degrade.

### Ubuntu 22.04 LTS image in Marketplace

Release 22.09.2 provides the Ubuntu 22.04 LTS OS image for download from
the zCompute Marketplace.

To download the image, in the UI go to **Machine Images** > **Marketplace**.

### Enhanced support utilities

This release contains enhanced utilities allowing Zadara Operations to
improve maintenance automation flows, time to resolution of incidents,
and to take more proactive measures to prevent incidents.
It is highly recommended to upgrade to this version to improve supportability.

### Notable fixes

* **NK-6332**:
  zCompute’s AWS-compatible `DescribeNetworkInterfaces` API returned the
  `status=in_use` (with an underscore) instead of `status=in-use`
  (with a hyphen).
  This has been resolved, in alignment with the AWS status value.
* **NK-6498**:
  An edge case was discovered where a user created incorrect values for
  DHCP options that caused the DNS to stop working.
  This has been resolved with a fix, preventing creation of incorrect
  values for DHCP options.
* **NK-3744, NK-3928**: Fix for Windows clock offset
  In previous versions, virtual machine migrations sometimes caused time
  changes in the guest. This is now fixed. In order for the fix to apply,
  users must set the Windows OS type in the VM properties, or use a
  Windows image for which the Windows OS type has already been set.

#### Security and non-functional fixes

* **NK-4847, NK-4848, NK-4406, NK-4785, NK-5238, NK-5293**:
  Some minor security and non-functional issues were resolved.

## Version 22.02

Release 22.02 is a major version extended by three service packs (SP1,
SP2 and SP3), that includes scalability, security, usability
improvements, new features and bug fixes.

The following section will breakdown the **main** additions and updates to the
Zadara Compute platform (zCompute).

### Premium instance types

Zadara now offers premium instance types based on Intel Gen 3 Platinum 8362.
Please contact sales for more information.

### NVMe instance types

Zadara now offers instance types with NVMe based on the premium types.
Please contact sales for more information.

### SAML SSO Support

Limited availability of SAML SSO support, for large installations.
Please contact Zadara ops/sales, if required.

### New zCompute Toolbox Fedora image including the V2Z utility (from Version 22.02-SP2)

A new zCompute Toolbox Fedora version is available.
It includes the following changes:

* Moved from CentOS 7.9 to Fedora 36.
* User changed from Centos to Fedora.
* New Symp client, compatible with Fedora.
* Now comes pre-installed with the V2Z tool for VMWare/Hyper-V VM imports.
* No default login user. Instead use the cloud-init procedure to change the
  user password
* Known issues:
  * **NK-3835**: The zCompute Toolbox Fedora shows the old version for members in the
    Marketplace. This means that tenants will still continue to download
    the older version if using the Marketplace.
    Instead look for a pre-downloaded image in the image list.
    If it’s not there, please ask Zadara ops or your cloud administrator
    to download it for you and make it publicly available in the system.
  * **NK-4055**: zCompute Toolbox Fedora is missing the Symp offline pip dependencies.
    This means that the python dependencies installation requires you to
    be online (internet) when running symp-update.

### Load Balancer API

Added compatibility with AWS fixed-response and redirect action types.

### Control over VPC subnet MTU

Users can now view and change MTU values per virtual network.

* The relevant networks are:
  : * Edge networks (for MSP administrators only)
    * Direct networks(subnets)
    * VPC subnets
* The minimum allowed MTU value is 1450 for standard VPC subnets or 1500
  for direct subnets or edge networks.
* The maximum allowed MTU value depends on the global MTU configured in
  the cluster (e.g. 9000 or 1500). If the global MTU configured in the
  cluster is 9000, the maximum allowed MTU value for all virtual networks
  will be limited to 8950.
* If a VM experiences MTU related connectivity issues and it resides on a
  public network (connected via a route-table to an internet-gateway),
  then it is recommended to set that network MTU to no more than 1500.
* **LIMITATION**: It is not possible to create a direct subnet in the GUI
  with desired MTU. However, the MTU can be updated post-creation.

![manage-edge-network](media/manage-edge-network.png)

![modify-direct-subnet](media/modify-direct-subnet.png)

![modify-subnet](media/modify-subnet.png)

### New Load Balancer Service Version

In order to support Kuberenetes integration with the managed load balancer
service, the functionality and APIs were enhanced to support fixed-response
and redirect action types that are compatible with their AWS equivalents.

* **Fixed-response**: Return a custom HTTP response
* **Redirect**: Redirect requests from one URL to another

![create-listener](media/create-listener.png)

* An example terraform script that create a Kuberenetes cluster that uses
  zCompute APIs for persistent volumes and a load balancer service type is
  available at: [https://github.com/Neokarm/neokarm-examples/tree/master/k8s](https://github.com/Neokarm/neokarm-examples/tree/master/k8s).
* **Known issues**:
  * **NK-2921**: Getting an **Action Failed** status when trying to create
    a listener in the ALB on an cluster upgraded to version 22.02-SP1,
    but with the previous LBAAS engine.

    **Workaround**: Upgrade the LBAAS engine before creating a new load balancer.
  * **NK-3247**: zCompute officially supports Terraform 2.47.
    Terraform AWS Provider versions higher than 3.33 fail on LBAAS ARN
    validation.

### Creating DNS VM in any VPC network (from Version 22.02-SP2)

In this release, Zadara introduces the option for the user to select the
subnet range to create a CoreDNS within a specified subnet range. This will
allow accessing the CoreDNS VM from outside the VPC when using a VPN or
VPC Peering, and forward DNS requests to it.

* By default, the DNS is created with IP address 169.254.64.253, which is
  only routable inside the VPC.

  ![compute-instances-coredns](media/compute-instances-coredns.png)
* In this release, the user will have the option to select from which of
  the VPC subnets to allocate the DNS IP.

  ![modify-vpc](media/modify-vpc.png)
* Users can modify a VPC and switch the VM DNS Subnet.
  This operation is mutli-step operation: First disabling the DNS, then
  changing the VPC network, and finally re-enabling the DNS.
  The GUI does this sequence automatically, so it is recommended to use the
  GUI for this operation.

### Custom policies support

IAM admins and tenant admins can now upload custom symp and AWS policies
and then apply them to users, using the new symp APIs or AWS API.

<!-- comment
<Add the API’s here from symp>

Or using AWS API

<Add examples here> -->
* Known issues:
  * **NK-3786**: Terraform is not supported at the moment, since it uses a
    the AWS CreatePolicyVersion API, which is not supported.
  * Creation of a new policy is only available via APIs and not
    from the GUI. Once the new policy has been created, the admin can apply
    it to users using the GUI.

### Fixes (Versions 22.02-SP2 and SP3)

Versions 22.02-SP2 and SP3 each include a collection of minor fixes.

### Notable fixes (Version 22.02-SP1)

* **NK-2374**: Fixed sporadic slow response times on volume operations.
* **NK-2767**: Fixed an issue with Windows VM sometimes hanging on migration.
* **NK-2814**: Fixed an issue with no connectivity between VMs with EIPs and
  other VMs with EIPs on the edge-network subnets. Effectively this meant
  that VMs between different VPCs could not connect over elastic IPs without
  Zadara ops intervention.
* **NK-3144**: Fixed issues on local routes delete in route tables, usually
  leading to UDP packets stop working after removing their elastic IP.

### Notable fixes (Version 22.02)

There is limited availability of version 22.02, and it should not exist
anymore in the field. All changes are listed under version 22.02-SP1.

## Version 21.12

Release 21.12 is based on the 21.10 version. The version includes various
core changes in order to significantly improve the guest operating system
performance (primarily Microsoft Windows).

## Version 21.10

Release 21.10 is a major release which includes scalability, security,
usability improvements, new features and bug fixes.

The following section will breakdown the **main** additions and updates to the
Zadara Compute platform (zCompute).

### Support multiple subnet ranges for Elastic IPs

It is now possible to add multiple non-contiguous subnets to the same Elastic
IP pool. Each may have its own gateway. This is aligned with the way site
operators purchase public IP ranges. As part of the change Zadara added the
following enhancement:

* The ability to add multiple subnet ranges to the same Elastic IP pool
* Each subnet may have its own default gateway
* Virtual Routers will not consume Public IPs anymore, instead a single
  public IP will serve the entire edge network. This IP is required to access
  external name servers.

Current limitations:

* Implicit DNAT Functionality is deprecated and will stop working in one of
  the next releases. This means: a VM without an EIP will not be able to
  reach an external destination. If such a flow is required, customers should
  use the explicit NAT GW service to implement this functionality. Also note
  that certain OS (e.g. Ubuntu, Windows) configuration for Zeroconf will
  overlap and interfere with the edge routers subnet address range - VM using
  DNAT cannot access VM using EIP.
* Can’t retire a subnet from the edge network without removing all the EIP
  from this subnet that is in use by instances. The Edge network DNS servers
  that are configured for the edge network must be accessible from all edge
  subnets If multiple edge subnets are configured, then all of them must be
  configured with the same DNS servers. configuring different DNS servers for
  different subnets will cause unexpected behavior
* These features require additional upgrade procedures for existing VPCs
  as external connectivity will be temporarily disrupted.

Site operators, for more info please contact Zadara support

### CentOS 8 has been added to the zCompute Machine Images Marketplace

Following customer requests, we added a link to download CentOS 8 to
our marketplace.

![cent-os8](media/cent-os8.png)

### Customize the system TLS certificate

MSPs that have cloud administration rights can now easily update the web
service TLS certificate via the API or UI, without the need to contact
Zadara Support.

Go to configuration -> Cluster certificates.
You’ll see the default self signed certificate
To upload the certificate just click the + button and upload the public
certificate and private key.

**Certificate** - The certificate file is the end-user certificate for your
domain which you received from the certificate authority. Drop the
certificate file in the designated area, -or- Browse to the location.

**Private Key** - The private key file is the file which contains the
private key which matches your certificate. Drop the private key file in the
designated area, -or- Browse to the location.

**Private Key Passphrase** - In case the private key is encrypted by
password, the passphrase should be specified in this field, otherwise the
field should be omitted.

On certificate creation the cluster will validate that the certificate is
not expired. After creating the certificate, it can be installed by
selecting it and clicking “Install”.

Known issues: The certificate can’t be updated while there are nodes in
maintenance or in failed states and therefore unreachable. There is a GUI
validation to prevent certificate change in these cases.

**Cluster certificate operation validates nodes are active before starting**

![cluster-certificate](media/cluster-certificate.png)

### Load balancer can be created without the internal Security Group

In the past, the system automatically opened the ports to any source.
This is very convenient for applications that are service web applications
but may be insufficient for tighter security control. Now, upon creation of
a Load Balancer instance the user can select whether to keep the mentioned
behavior, or like in other clouds, to have full control of the security
group by attaching user controlled security groups for the instances.
Note: If configured without an Internal Security Group controlled by the
platform the access to listener ports is blocked by default. The user must
add their own security groups to enable traffic.

### The granularity of the protection plan can now be set in minutes

![protection-group](media/protection-group.png)

### Force delete VM snapshot with timer based on snapshot creation

The UI now contains a force delete button action to terminate stuck
snapshots. Note that when creating a snapshot between different storage
pools is an operation that may take a long time. To prevent premature
attempts to delete snapshots that may succeed the button is grayed out
until a 10 minute timer has been completed.

![delete-snapshot](media/delete-snapshot.png)

### Clear indication added for incompatible instance sizes

Users are now allowed to create an instance type of any size, even when
the size is larger than the node’s size (RAM or CPU). However, both the UI
and API indicate the instance type is incompatible with the hardware size.

### lbaas-manager and Core DNS image updates

A new load balancer and core DNS engine images have been created to account
for bug fixes that were carried out. Once the first phase of the upgrade
is done, the admin needs to go to the upgrade panel and click upgrade all
engines (no screenshot attached). Once image upgrade is done and the
engines are updated all the users that have LBaaS and core DNS instances
will see that updates are ready

![service-upgrades](media/service-upgrades.png)

![upgrade-status](media/upgrade-status.png)

![load-balancer](media/load-balancer.png)

### DHCP to Pass user configured DNS when default VPC is disabled

By default the system created a VPC DNS service which is a forwarding DNS
for the user’s configured DNS in the DHCP options while at the same time it’s
the authoritative DNS for the VPC entities. Hence the instances are configured
to point to the VPC DNS. Before the change, if the user chose not to use
VPC DNS (example, when using an Active Directory) then the system did not
pass through the user configured name server directly to the VMs as the user
expected. Now when the user disables the VPC DNS the system will provide
the configured DNS all the way to the VM via DHCP.

### Direct-Subnet allocation pools can be updated while attached

It is now possible to update allocation pools while the direct network is
attached.

### Fixed an issue where trying to restore from previous version snapshot failed

Due to fields that were updated in later releases, VM snapshots restored
from previous versions that didn’t include the os_type field failed.
This has now been addressed.

### After refresh on the account page, the Enforce MFA appeared unenforced

**What was fixed**

The status now always reflects the actual setting

### After a long network disconnect, DNS and LBaaS go into an error state

**What was fixed**
Fixed issue where after a long disconnect, managed services fail to refresh
state and lose the state. Relevant for CoreDNS (VPC) and LBaaS

**Workaround for pre-R8**
In order to fix it in the field, identify the VMs supporting the
service and restart the service VMs from the symp CLI.

## Version 21.4

### VPSA based VM and Volume DR

zCompute can now orchestrate the protection and restore of individual Instances
and Volumes between two zCompute systems connected to different VPSAs.
The data transfer is done using VPSA volume mirroring between sites, and
zCompute has enhanced the protection plans capability to support setting of
mirroring on the volumes and to place the required metadata for restoration
on an NFS share on the destination VPSA.

### Self service password reset

Users can reset their passwords directly from the login page using the “Forgot
your password?”.

### Added support for virtual network appliances within a VPC

Some virtual network appliances are either a virtual firewall that requires
full control of the traffic coming in/out of the interface, or may be a VPN
appliance that has extra IPs representing the client endpoints.

To support the above, there are now two new settings:

1. Enable/Disable Source/Dest Check - which only controls the L3 IP check.
   Item 3 above. (This operation can be done by any tenant member)
2. Enable/Disable Port security. This one can only be set by MSP or Cloud
   admins.

The options were added to the security groups settings of the interface

![source-dest-check](media/source-dest-check.png)

### Improve VNC console security

Starting of R7 version, users must explicitly allow Zadara administrators or
MSP administrators to connect to the VM VNC, otherwise only users from the
same project can access the VM VNC.

### Auto-populate instance name as a DNS record

When creating a new instance from the Management user interface, the DNS
name field in the VM creation wizard will auto populate the VM name as the
DNS name.

### VPC deletion wizard

A VPC is a construct with many sub resources and/or attached resources.
A VPC deletion wizard was added to easily delete a VPC and it’s related
resources as long as there are no running workloads in the VPC.

### Allow user to set VPC as default VPC

The user is now able to change the default VPC by setting a new VPC as the
default one.

### Additional changes and bug fixes

- **NK-1103** - Clone image operation to allow creation of an image with IDE
  devices or extra data disks
- **NK-1082** - Add the option to export filtered event list to CSV from the
  management user interface
- **NK-1108** - Delete project now deletes all the default resources created
  with the project
- **NK-797** - Fixed an issue where Compute Overview’s “Top VMs” sort worked only
  with CPU

## Version R5 (V21.1.0)

- **NK-227** - Managed resources (like: ENI, EIP, Security Groups,etc…)
  are now hidden by default and need to explicitly toggle to see them in all
  views
- **NK-294** - Added a capability to the GUI to change Direct Network IP
  addresses allocation pool
- **NK-470** - An issue if a wrong error when extending volume on a live
  instance was resolved
- **NK-502** - VM snapshots are now restored to the requested pool
- **NK-594** - MFA is now supported using authenticator application on a
  mobile device (e.g. Google Authenticator), for both Console and CLI login
- **NK-771, NK-777** - The system now allows admins and tenant_admins to
  create instance types and also specify the CPU topology over sockets
- **NK-795** - A network mini-graph was added on the nodes list view
- **NK-809** - Added clear indication and system alarm to the cloud admin
  when running out of EIP per IP pool
- **NK-831** - Prevent deletion of ENI from the GUI
- **NK-865** - Ability to export filtered lists as CSV file was added to all
  GUI tables
- **NK-897** - GUI and Sym/AWS web shell now work even if firewall is
  forwarding different public port to 443
- **NK-904** - Snapshots and volumes are now deleted when deleting image
  from the marketplace
- **NK-906** - Tenant Admin gets all the permissions of a member
- **NK-913** - Images and Marketplace Images now have an Operating system
  field The Operating system is inherited by the Instances created out of
  these images
- **NK-955** - The reboot button was removed from the instance view
- **NK-967** - A new inspector command was added that can be used to validate
  the MTU of all network interfaces after changing the MTU in the cluster

## Version R4 (V5.5.4)

### Improvements to the VM Creation dialogue

NK-733 - Instance creation wizard Steps are according to functionality

Step1 - Compute

Step2 - Storage

Step3 - Network

Step4 - Config

![image42](media/VMWizard.jpg)

### Show the VPC virtual router IP on the direct subnet

NK-668 - It is, now, easier to set the static routes in the external routers in case that VMs on non
direct VPC subnets need to be accessed via the direct subnets or need to access entities on the direct
subnet that are outside of the direct subnet allocation pool

![image41](media/DirectSubnet.jpg)

### Additional changes & bug fixes

- **NK-234** - An issue, where direct-network was not listed as route in
  route-table, was fixed
- **NK-444** - Improved Resource Consumption tile to make it easier to understand
- **NK-546** - create default VPC operation was added to the GUI
- **NK-621** - When attaching Elastic IP, if there is only one network
  available, it’s now selected automatically
- **NK-648** - The issue of Load Balancer creation with elastic IP that belong
  to an instance was resolved
- **NK-661** - Basic event filtering was fixed, and now working as expected
- **NK-663** - Project delete is now possible even if a NAT gateway still
  exists in the project but in a deleted state
- **NK-681** - AWS compatible API now support import from snapshot
- **NK-703** - CoreDNS VMs removed from the network topology diagram. Clicking
  the VPC shows if DNS is enabled
- **NK-729** - When creating key pair for a VM the keys are automatically
  download, to prevent inaccessible instance
- **NK-730** - Neokarm Policies were renamed Symp Policies
- **NK-739** - Cleanup of aging images from the marketplace
- **NK-800** - The “default” security group is properly displayed even if a
  load balancer is configured in the same VPC

## Version R3 (V5.5.3)

### Quick start button and screens

Look for the quick start button on the main navigation bar

![image1](media/QuickStart.jpg)

It will easily guide you via the most common tasks such as creating VM instances, attaching storage volumes, etc…

![image2](media/QuickStartMenu.jpg)

### New VPC creation wizard

A wizard was added to assist you with creating the 3 common topologies of VPCs
with private or public subnets

![image3](media/VPCWizard.jpg)

The Wizard can activated from the Quick Start and from the VPCs pane

![image4](media/VPCs.jpg)

### Elastic Network Interfaces

A new screen was added to manage Network Interfaces.
Network interfaces may be attached and detached from an instance and may even move between instances or be a route target.
Until now, you could only create it implicitly when connecting an instance to a subnet.
Now it has its own screen that exposes the full functionality.

A user can now Create/attach/detach/add security groups and update the name of any network interface.

![image5](media/NetworkInterfaces.jpg)

### Toolbox

A VM with client tools available on the marketplaces
Toolbox v5.5.0. The images contain the Symp CLI, AWS CLI, Python 2.7, Boto3 and Terraform.
The instance should be created with an SSH key pair.

### Additional changes & bug fixes

- **NK-386** - Added Storage Pool update mechanism
- **NK-405** - The GUI was fixed to allow subnet name change
- **NK-422** - The GUI was fixed to enable detach security groups from a
  network interface
- **NK-467** - Account members can now generate Access keys
- **NK-468** - All the networking related tabs are now merged into the networks
  tab Under instance view. EIP, Security groups and Interfaces operations
  are done from the same place.
