Resources Management¶
Scope: Object Storage Administrator
Monitoring Drives¶
To monitor drives in your Object Storage system, open Resources > Drives.
Viewing Drives Properties¶
The Drives details (properties and metering), are shown in the South Panel tabs:
Properties
Each Drive includes the following properties:
Property |
Description |
|---|---|
ID |
An internally assigned unique ID |
Name |
Automatically assigned name. |
Capacity |
The Drive capacity in GiB |
Storage Node |
The Storage Node that contains the selected Drive |
Virtual Controller |
The virtual controller that owns the selected drives and performs IO operations on it |
Storage Policy |
The Storage Policy where the selected Drive belongs |
Fault Domain |
The Zadara cloud Fault Domain this Drive resides belongs to |
Protection Zone |
The Zadara cloud protection zone this drive is physically located at |
Type |
Drive type: SATA, SAS, SSD |
UUID |
The unique identifier of the drive |
Status |
|
Added |
Date & time when the drive was added |
Modified |
Date & time when the drive was last modified |
Disk Metering
The Metering Charts provide live metering of the IO workload associated with the selected Drive.
The charts display the last captured performance samples. An interval length can be one of the following: 10 second, 1 minute, 10 minutes, 1 hour, 1 day, 1 week.
The Auto button allows to switch to continuously-update live metering info.
The following charts are displayed:
Chart |
Description |
|---|---|
IOPs |
The number of read and write commands issued to the selected Drive per second |
Bandwidth (MB/s) |
Total throughput (in MB) of read and write commands issued to the selected Drive per second |
Latency (ms) |
Average response time of all read and write commands issued to the selected Drive per selected interval |
Backend Metering
The following charts are displayed:
Chart |
Description |
|---|---|
Throughput (OP/s) |
The number of operations (PUT/GET/DELETE) that were sent to the selected Drive per second |
Bandwidth (MB/s) |
Total throughput (in MB) of read and write commands that were sent to the selected Drive per second |
Latency (ms) |
Average response time of all operations (PUT/GET/DELETE) that were sent to the selected Drive per selected interval |
Monitoring Virtual Controllers¶
Virtual Controllers are Virtual Machines running on the Zadara zStorage cloud that serve client operations on the Object Storage. For a full list of the VC responsibilities refer to Virtual Controller. Virtual Controllers are automatically created and added/removed to the Object Storage configuration, depending on the number of the allocated drives. In case the workload have changed and additional performance are required - the object storage owner can add Proxy only VCs from the Zadara Provisioning Portal as described in Adding Proxy Virtual Controllers.
Viewing VCs Properties¶
Properties
Each Virtual Controller has the following properties:
Property |
Description |
|---|---|
ID |
An internally assigned unique ID |
Storage Role |
Proxy+Storage / Proxy-Only |
Management Role |
|
Status |
|
Storage Node |
The Storage Node hosting selected VC |
Fault Domain |
The Zadara cloud Fault Domain this VC resides belongs to |
Frontend IP |
The IPv4 or IPv6 address allocated to the VC |
Backend IP |
The VC IP address on the backend network that connects to the Drives |
Added |
Date & time when the VC was added |
Modified |
Date & time when the VC was last modified |
Drives
List the drives assigned to the selected Storage Policy.
Virtual Networks
A set of available IP addresses within a specific network segment. Virtual networks are allocated for a specific cloud tenant and within a specific avilable cloud VLAN.
System Usage
This chart shows the CPU utilization of the selected VC.
Backend Metering
The Metering Charts provide live metering of the IO workload at the backend of the selected VC.
The charts display the metering data as it was captured in the past 20 intervals. An interval length can be one of the following: 10 second, 1 minute, 10 minutes, or 1 hour, 1 day, 1 week. The Auto button lets you see continuously-updating live metering info.
The following charts are displayed:
Chart |
Description |
|---|---|
IOPS |
The number of operations (PUT/GET/DELETE) issued to objects and handled by the selected VC per second |
Bandwidth (MB/s) |
Total throughput (in MB) of read and write commands issued by the selected VC per second |
Latency (ms) |
Average response time of all operations (PUT/GET/DELETE) issued to objects and handled by the selected VC per selected interval |
Account Metadata Metering
The Metering Charts provide live metering of the IO workload on the accounts database at the backend of the selected VC.
The charts display the metering data as it was captured in the past 20 intervals. An interval length can be one of the following: 10 second, 1 minute, 10 minutes, or 1 hour, 1 day, 1 week. The Auto button lets you see continuously-updating live metering info.
The following charts are displayed:
Chart |
Description |
|---|---|
IOPS |
The number of operations (PUT/GET/DELETE) issued to the accounts database and handled by the selected VC per second |
Latency (ms) |
Average response time of all operations (PUT/GET/DELETE) issued to the accounts database and handled by the selected VC per selected interval |
Container Metadata Metering
The Metering Charts provide live metering of the IO workload on the containers database at the backend of the selected VC .
The charts display the metering data as it was captured in the past 20 intervals. An interval length can be one of the following: 10 second, 1 minute, 10 minutes, or 1 hour, 1 day, 1 week. The Auto button lets you see continuously-updating live metering info.
The following charts are displayed:
Chart |
Description |
|---|---|
IOPS |
The number of operations (PUT/GET/DELETE) issued to containers and handled by the selected VC per second |
Latency (ms) |
Average response time of all operations (PUT/GET/DELETE) issued to containers and handled by the selected VC per selected interval |
Object Metadata Metering
The Metering Charts provide live metering of the IO workload on the object database at the backend of the selected VC .
The charts display the metering data as it was captured in the past 20 intervals. An interval length can be one of the following: 10 second, 1 minute, 10 minutes, or 1 hour, 1 day, 1 week. The Auto button lets you see continuously-updating live metering info.
The following charts are displayed:
Chart |
Description |
|---|---|
IOPS |
The number of operations (PUT/GET/DELETE) issued to objects and handled by the selected VC per second |
Latency (ms) |
Average response time of all operations (PUT/GET/DELETE) issued to objects and handled by the selected VC per selected interval |
Frontend Metering
The Metering Charts provide live metering of the IO workload at the frontend of the selected VC.
The charts display the metering data as it was captured in the past 20 intervals. An interval length can be one of the following: 10 second, 1 minute, 10 minutes, or 1 hour, 1 day, 1 week. The Auto button lets you see continuously-updating live metering info.
The following charts are displayed:
Chart |
Description |
|---|---|
Throughput (OP/s) |
The number of operations (PUT/GET/DELETE) issued to objects and handled by the proxy of the selected VC per second |
Bandwidth (MB/s) |
Total throughput (in MB) of read and write commands issued to proxy of the selected VC per second |
Latency (ms) |
Average response time of all operations (PUT/GET/DELETE) issued to objects and handled by proxy of the selected VC per selected interval |
Managing Storage Policies¶
The Storage Policy provide a way for object storage providers to differentiate service levels, features and behaviors of an Object Storage deployment.
Policies can be thought of as a group of drives, with a redundancy level policy assigned to it.
Before placing object data into the Object Storage, users create a container which holds the listing of all objects stored under the container’s namespace. Users can select the Storage Policy that will be used when storing data objects under a container’s namespace when they create the container. All objects stored in a container will be placed according the configuration of the Storage Policy which was set upon the Object Storage creation.
To ensure availability of the Object Storage data, the drives assigned to a Storage Policy are evenly distributed between Object Storage Fault Domains. The cloud administrator defines the Fault Domain of each Storage Node. The system allocates drives across zones, based on the Storage Policy type.
Storage Policies allow objects to be stored based on the following criteria:
Quality of Service: By using different disk drives for different policies, tiers of storage performance can be created. For example, an SSD-only policy can be created and used to implement a low-latency/high performance tier.
Number of Replicas:
2 Way replication offers protection for one FD failure, at the cost of 50% storage utilization.
Erasure Coding (4+2) offers protection for one FD failure, at the cost of 67% storage utilization.
The following Storage Policies are supported:
Object Storage Data Protection Policies¶ Policy Type
Redundancy
Minimal Configuration
2 Way
x 2
2 Storage VCs on 2 SNs
Erasure Coding 4+2
x 1.5
3 Storage VCs on 3 SNs
Each drive in the system is assigned to one Storage Policy.
Object Storage is created with a default data Storage Policy for objects, and another system Storage Policy for metadata. The Object Storage administrator (zios_admin) can later expand/shrink the storage policy based on their needs.
Storage Policies¶
When the Object Storage system is created, 2 policies are created by default:
MetadataPolicy: Used to store the Accounts and Containers’ metadata
2-way/EC-protection-policy: Used to store the users’ objects, usually contains all the drives that were assigned to the Object Storage at creation time, with 2-way replication or EC protection, according to the initial selection on the provisioning portal.
Policies Properties¶
You can view the following properties and metering information in the Policies Details south panel tabs:
Properties
Each Policy includes the following properties:
Property |
Description |
|---|---|
General |
|
ID |
An internally assigned unique ID |
Name |
The name of policy |
Type |
Object or Account/Container |
State |
Not Configured / Configuring / Initialized |
Description |
A user defined policy description |
Ring Version |
Ring Database version |
Default |
Yes/No |
Redundancy Level |
2-way/EC |
Policy Capacity |
Total usable capacity of the storage policy |
Policy Used Capacity |
Total capacity used in the storage policy |
Data Usage |
Amount of written data in the storage policy by the Object Storage account’s users |
Containers Count |
Total amount of containers created within the policy |
Objects Count |
Total amount of objects created within the policy |
GB per Month Price |
Price of used capacity for charge back purposes |
Added |
The date and time when the policy was added |
Modified |
The date and time when the policy was last modified |
Policy Health |
|
Health Status |
Normal / Degraded / Critical |
Balance |
Indicates the progress of the rebalance process |
Rebalance Paused |
Yes/No |
Last Data Rebalance |
The date and time of the last data rebalance |
Last Ring Rebalance |
The date and time of the last ring rebalance |
Note
Please note that Policy Used Capacity attribute will always reflect higher capacity consumption than Data Usage. The Policy Used Capacity also include the underlying object allocated metadata (4K) and the actual object storage allocation (the minimal allocation unit is 4K). Data Usage is the actual account’s data usage representation.
Drives
List the drives assigned with the selected Storage Policy.
Capacity Metering
The Metering Charts provide live metering of the capacity usage associated with the selected Policy.
The charts display the metering data as it was captured in the past 20 intervals. An interval length can be one of the following: 10 minutes, or 1 hour, 1 day, 1 week. The Auto button lets you see continuously-updating live metering info.
The following charts are displayed:
Chart |
Description |
|---|---|
Used Capacity |
Total storage capacity consumed in the selected policy |
Containers |
Total numbers of containers that store their objects in the selected policy |
Objects |
Total numbers of objects stored in the selected policy |
Backend Metering
The Metering Charts provide live metering of the IO workload associated with the selected Policy.
The charts display the metering data as it was captured in the past 20 intervals. An interval length can be one of the following: 10 second, 1 minute, 10 minutes, or 1 hour, 1 day, 1 week. The Auto button lets you see continuously-updating live metering info.
The following charts are displayed:
Chart |
Description |
|---|---|
Throughput (OP/s) |
The number of operations (PUT/GET/DELETE) issued to the Drives of the selected policy per second |
Bandwidth (MB/s) |
Total throughput (in MB) of read and write commands issued to the Drives of selected policy per second |
Avg. Drive Latency |
Average response time of all operations (PUT/GET/DELETE) issued to objects in the selected policy per selected interval |
Frontend Metering
The Metering Charts provide live metering of the IO workload associated with the traffic coming to the selected Policy.
The charts display the metering data as it was captured in the past 20 intervals. An interval length can be one of the following: 10 second, 1 minute, 10 minutes, or 1 hour, 1 day, 1 week. The Auto button lets you see continuously-updating live metering info.
The following charts are displayed:
Chart |
Description |
|---|---|
Throughput (OP/s) |
The number of operations (PUT/GET/DELETE) issued to objects in the selected policy per second |
Bandwidth (MB/s) |
Total throughput (in MB) of read and write commands issued to the selected policy per second |
Avg. Latency (ms) |
Average response time of all operations (PUT/GET/DELETE) issued to objects in the selected policy per selected interval |
Capacity Alerts¶
The Object Storage administrator can set their own custom configuration for capacity monitoring.
Alert Threshold
Send alert when it is estimated that the policy will be at full capacity within the given time period (default 43,200 minutes - 30 days).
Alert Interval
The period of the sample collection window to calculate the rate at which the free space is consumed in the storage policy.
Default 1,440 minutes
Minimal value is 10 minutes
Minimum used capacity to trigger the alert is 60%
Emergency Threshold
The threshold to trigger administrator capacity emergency state notifications
Default is 85%
Minimal value is 60%
Alert will be triggered hourly as long as the policy is in “Emergency state”
Adding Drives to Policy¶
Drives are added to an Object Storage policy via the Provisioning Portal. To add drives into a policy, go to the Provisioning Portal, select the Object Storage of interest and click Add Storage. Follow the instruction here: Adding drives.
Note
Drive-related operations in a storage policy will require rebalance that might take several hours/days until completion.
Removing Drives from Policy¶
If there is a need to reduce the total available capacity of a given policy, or to remove some failed drives that were detached from the policy, you may remove drives from the policy and return them to the cloud for a different use. To remove drives from a policy, go to GUI > Policies, select the policy of interest, and click Remove Drives.
The dialog that opens will list all the drives types and quantities that currently belong to the policy. The system support removing a pair of drives in a single operation. In case a larger volumes of drives needs to be removed please contact Zadara’s support.
Note
Drive-related operations in a storage policy will require rebalance that might take several hours until completion.