Hyperdisk Storage Pools, the first block storage solution from a hyperscale cloud provider to allow thin-provisioning, data reduction, and capacity pooling, will be generally available, as Google cloud revealed at Google Cloud Next 2024. By reducing your Total Cost of Ownership (TCO) by up to 30–50%, Hyperdisk Storage Pools enable you to streamline block storage management, modernise SAN-based operations, and increase efficiency. Storage Pools are available right now via the Google Cloud console. Let’s examine Hyperdisk Storage Pools’ operation and integration into your environment in this blog post.
Hyperdisk Storage Pools can store big amounts using Compute Engine. A Hyperdisk Storage Pool provides pre-purchased capacity, throughput, and IOPS to applications as needed. Create, manage, and use discs in pools with Hyperdisk Storage Pools for multiple workloads. Manage discs in bulk to save money and increase capacity and performance. By employing exactly the storage you need in Hyperdisk Storage Pools, you simplify capacity forecasts and decrease management by going from hundreds of discs to one pool.
Benefits of storage pools include:
- Thin provisioning and data reduction enable Hyperdisk Storage Pools store data efficiently and achieve best-in-class TCO.
- Hyperdisk Storage Pools take advantage of thin provisioning and data reduction to maximize resource utilization and minimize TCO.
- Management overhead reduced Higher Flexibility – Hyperdisk Storage Pools allow workload owners to provide larger discs and only use what they need, eliminating capacity and performance forecasts and rescaling downtime.
- Workloads use Hyperdisk volumes the same way with storage pools. No downtime or workload disruptions are needed.
Data on discs in a storage pool remain isolated as if they were not.
Use storage pools
Storage pools solve these issues:
Trouble predicting resource requirements when transitioning on-premise SAN workloads to Google Cloud.
Estimating application performance and capacity can take weeks and be error-prone, delaying a cloud migration or application rollout.
In Hyperdisk Storage Pools, you can overestimate the capacity needed to make discs and then only use the disc space you write data on.
Underuse of resources
Ensuring volume utilization is difficult and painful. Block storage is often underutilized because peak capacity and performance must be provided to avoid outages and slowdowns. However, few applications reach those levels.
Using Hyperdisk Storage Pools, you establish a pool to meet workload capacity needs. To keep utilization below 80%, the Hyperdisk Storage Pool automatically adds capacity.
Complex workload block storage management
Managing hundreds or thousands of disc volumes takes time and resources away from innovation.
When creating VM discs in a storage pool, you can choose a size bigger than expected workload. Workloads write to discs, reducing storage pool capacity. You only subtract the block storage you use from the storage pool capacity, not the disc creation amount. The disc size you choose limits capacity. Only change the disc size if this limit is reached.
If you build discs in the storage pool and your workload exceeds capacity planning over numerous discs, you can expand capacity. All discs in the storage pool can use the extra capacity.
Hyperdisk Storage Pools options
Features of storage pools include:
- Allocating blocks as needed instead of all at once is capacity thin provisioning. This prevents low storage utilisation, where lots of disc space is assigned but not used.
- Data reduction: Storage pools improve efficiency with various data reduction technologies. Data type strongly affects data reduction. Data compressed or encrypted before storage in a Hyperdisk Storage Pool won’t reduce.
- To prevent failures caused by insufficient capacity, Hyperdisk Storage Pools automatically adds capacity when utilisation surpasses 80% of provisioned capacity.
Hyperdisk Storage Pool operation
A storage pool with the aggregate capacity and performance your workloads need is created, then discs are added. Connect the discs to your VMs. Create discs with a larger size than needed. This allows future growth without disc resizing.
Only the disc space your workloads require after data reduction is withdrawn from the Advanced capacity storage pool if the disc was generated there.
To increase disc capacity, increase the storage pool’s provided capacity. The discs’ disc space can then rise to the size you set when creating them. By generating discs in an Advanced capacity storage pool with a big initial size and allocating additional space, you consolidate disc storage administration and decrease costs.
Storage pool capacity is managed. Increase or decrease a storage pool’s provided capacity to change disc capacity. If the storage pool’s usage capacity or the aggregate capacity of all discs reaches 80% of its specified capacity, an advanced capacity storage pool automatically adds capacity. If auto-grow fails, you can manually add storage pool capacity up to 1 PiB.
If a storage pool achieves 100% utilization and has no free space, writes to all discs fail unless data or discs are deleted. Most commercial software treats full disc write errors like hardware failures.
To avoid out-of-space issues, actively maintain and monitor your storage pool. You should also know how your workload will react to a disc out of space issue in a storage pool.
Hyperdisk Storage Pool provisioning
You can configure a Hyperdisk Storage Pool with Standard or Advanced capacity.
Storage pools standard capacity
Standard capacity provisioning creates discs in the storage pool until all discs exceed the pool’s capacity. Standard capacity storage pool discs use capacity like non-storage pool discs.
Storage pools with high capacity
Advanced storage pools allow thin-provisioning and data reduction for capacity, allowing you to provision discs with more capacity than purchased. Advanced capacity storage pools use capacity based only on bytes written to discs after data reduction, so you can give end users and applications more capacity than you paid.
Advanced capacity lets you build discs in the storage pool that exceed its provisioned capacity by up to 500%. Data written determines storage pool capacity, not disc capacity. Advanced capacity storage pool discs use capacity differently from Standard capacity and non-storage pool discs.
You can fill discs in an Advanced capacity storage pool to their allotted size if the data written to all discs doesn’t exceed the capacity. The auto-grow feature adds capacity to the storage pool if utilization reaches 80% of provisioned capacity. If the storage pool is full, writes to all discs will fail until you delete data or discs to reduce its capacity. Since most software applications treat writes to full discs as hardware failures, it’s important to both:
- Monitor your storage pool to minimise disc space shortages.
- Know how your workload reacts if it does.
Remember that storage pools can’t see your file system. Deleted data is still in use until your OS marks it unused using DISCARD or TRIM. Most third-party OS images and all Google-provided OS images do this by default, but you should confirm this if you are not using one. Find out how to verify or configure this feature at Disable lazy initialization and enable DISCARD commands.
Hyperdisk Storage Pool Types
The discs you can generate in a Hyperdisk Storage Pool depend on its type.
- Hyperdisk Throughput Storage Pool: You choose capacity and throughput while creating the storage pool. Created Hyperdisk Throughput discs in the storage pool use some allotted capacity and throughput.
- Hyperdisk Balanced Storage Pool: You set capacity, throughput, and IOPS while constructing the storage pool. You employ some of the storage pool’s capacity and performance to make Hyperdisk Balanced discs with capacity and performance above baseline.
High-throughput storage pools
Storage Pools can manage Hyperdisk Throughput disc utilization.
Performance
Storage pooled discs perform the same as non-pooled discs.
Pricing
Capacity, throughput, and IOPS determine Hyperdisk Storage Pool pricing.
Standard capacity storage pools price capacity like discs. Standard capacity in a Hyperdisk Balanced Storage Pool costs the same as standalone capacity.
Thin provisioning and data reduction cost more, hence advanced capacity is more expensive. Despite this premium, thin-provisioning and data reduction can lower block storage costs by increasing efficiency and utilization.
