There is a significant amount of confusion among buyers about storage scalability, elasticity and on-demand elasticity. Ambiguities include whether scalability includes capacity and performance or just capacity. It can also be unclear whether on-demand storage elasticity is automated or manual, scales up and down, or just scales down.
Marketing use of the same words to describe dissimilar capabilities helps create uncertainty and obfuscate the differences. As a result, storage admins should understand how each are separate but related concepts and capabilities. They are not mutually exclusive. However, many storage systems and cloud services do not provide all of them.
Storage admins must know the minimum requirements for any system they expect to use. They must also know what capabilities they want but can grudgingly live without. For example, nondisruptive storage elasticity may be bottom-line essential, while automated storage elasticity may be desirable and a key differentiator between different services.
What is storage scalability?
Storage scalability measures capacity and performance.
Capacity scalability is how much data the storage system or service can address, manage and support with acceptable performance.
There are several storage systems and services that can provide a lot of capacity but lack in performance. Think storage archives, such as tape libraries, optical storage and some object stores.
There are others that can address, support and manage capacity and maintain acceptable performance, regardless of the scale. These systems and services may use flash, storage class memory and high-performance interconnect.
Still others can optionally scale capacity, performance or both separately -- an important aspect of many scale-out systems and services.
Performance scalability is measured in IOPS; it is articulated as megabytes per second, gigabytes per second or even terabytes per second. Scaling performance -- sometimes automatically -- without scaling capacity may be a critical capability. Peak online transaction processing usage is a prime example, especially during an unexpected demand spike. Online analytical processing, machine learning and deep machine learning all demand a lot of throughput for a short period.
Storage can scale up, scale out or do both. Scaling up tends to be capacity only, for a given set of storage controllers, typically one or two. The capacity scalability is the maximum amount of storage capacity supported. Scaling out is the ability to add storage controllers with or without additional capacity. The scale-out capacity and performance are limited by the number of storage controllers or nodes the vendor has tested and supports. Some storage scalability is both up and out.
What is storage elasticity?
Elasticity is an entirely different storage concept. It originated with grid technologies, server virtualization and cloud services. Storage elasticity refers to the ability to respond to variable workload changes by allocating and deallocating resources as each application demands. It is here that confusion arises.
That allocation and deallocation can occur in real time based on defaults, preestablished policies or AI -- without human intervention. This is known as on-demand elasticity. It can scale up and down automatically. Performance is the key metric that scales up and down.
Once a system consumes capacity, it is generally no longer available to scale down. Ephemeral container storage is a notable exception. The lack of human intervention is the part that gets conflated.
There are several storage systems and cloud storage services that enable some of their storage to scale on demand elastically. Others require manual intervention, which may be as simple as a few clicks on a screen to change a hardware shape, but it is still manual and not in real time. Scaling down is the same. It is still storage elasticity, but it is not on-demand storage elasticity.
For the most part, storage resource demand is a lumpy, nonlinear process with imperfect predictability; there are always ebbs and flows. Not all applications require peak performance all the time. Some applications may require peak resources at the end of a quarter or during the early morning hours. Others may not require peak resources except during a specific quarter during the year, such as with retail. Elasticity enables the system to respond to the lumpiness of the demand cost-effectively.
Benefits of storage scalability and elasticity
When a storage system does not have elasticity or on-demand elasticity, the storage admin must plan for the worst and build out that system for the estimated peak of demand for all applications concurrently. Doing so enables smooth operations during peak demand, but admins must overprovision and buy excessive processing, memory, cache and capacity. This process is inefficient and costly. The estimation may be way off as well. As a result, many storage admins significantly overestimate that peak demand. It's also why cloud storage services have become so attractive.
Storage scalability, elasticity and on-demand elasticity are software features built into the storage software. There is an emerging trend, which started in public cloud services, of abstracting the storage services -- including scaling, elasticity and on-demand elasticity -- from the underlying physical storage. Doing so frees the data from being locked in to a given vendor on premises or in the cloud.
Dig Deeper on Storage architecture and strategy
Related Q&A from Marc Staimer
Network File System and Common Internet File System/Server Message Block were designed to work with any operating system, but NFS remains dominant in... Continue Reading
Latency in object stores that stems from a large amount of metadata means the technology is better suited to non-transactional data. Continue Reading
Eventual consistency in object stores can be an issue because object storage is spread over many nodes and up-to-date data may not always be ... Continue Reading