Energy efficient storage architecture in the AI era

How AI workloads are driving data center power concerns

Historically, one of the biggest challenges for data center operators has been accommodating all of an organization's IT resources without running out of floor space. After all, nobody wants to shoulder the cost of a data center expansion or the construction of a new facility. Today, however, the availability of sufficient electricity is becoming a bigger operational constraint than floor space.

AI servers consume far more power than the servers used to host traditional workloads. According to some estimates, "Traditional data centers operate at 5-10 kW per rack, while AI-optimized facilities now require 60+ kW per rack within the same square foot footprint." Most of the datacenters built before the AI boom were never designed to accommodate such demand, and utility expansions can take years to complete.

Simply put, power has gone from being an operating expense to becoming a business constraint. Hence, organizations must take immediate steps to ensure that they are using the available power efficiently. Every watt of power that is being consumed by storage resources is a watt that cannot be allocated to the AI infrastructure.

What Does Storage Actually Contribute to Your data center Power Bill?

Like any piece of data center hardware, storage arrays consume power. Even a relatively small storage appliance can easily consume over 100 watts. Of course, larger arrays consume more power. According to Solved Magazine, a 2U SAN might contain 20 to 40 drives and consume 2 to 5 KW of power. As an example, the Dell PowerMax 8500's power consumption varies based on its configuration. However, the smallest number given by Dell is 7.124 kVA (approximately 5.6 KW).

If these numbers sound high, it's worth remembering that in an enterprise-class environment, storage hardware does far more than just hosting drives. Functions such as erasure coding and encryption consume significant compute resources, which in turn increase power consumption.

It's also worth considering that the storage hardware itself is only half of the story. As the storage hardware consumes power, it releases heat as a byproduct. This thermal load forces cooling systems to work harder, further increasing power consumption. In other words, every watt consumed becomes heat that has to be removed.

Intelligent Tiering as an Energy Strategy

AI workloads generally require access to vast inference, analytics or training datasets amounting to many terabytes (TB) or even petabytes, so the need for storage hardware certainly isn't going away. Even so, it may be possible to drive down costs through intelligent storage tiering.

Storage tiering was introduced at a time when flash storage offered extremely limited capacity with a very high cost per gigabyte. The idea was that not all data requires the same level of performance. Hence, frequently accessed "hot" data could be placed on high-performance SSDs, while occasionally used "warm" data could reside on less expensive, higher-capacity spinning media. Rarely accessed "cold" data could be moved to cheap archival disks or even kept offline.

As flash storage prices began to decline, many organizations moved their performance-sensitive workloads to all-flash arrays, with traditional HDD/SSD tiering being used less frequently for primary workloads. However, intelligent tiering is beginning to see a resurgence thanks to the need for reducing power consumption in the data center.

Simply put, higher-performance storage tends to use more power, and because it is consuming more power, it also produces more heat. This is not necessarily to say that an SSD uses more power than a spinning disk. In some cases, an SSD might actually use a bit less power than spinning media.

What needs to be considered, however, are metrics such as watts per IOPS, watts per transaction, watts per TB, or watts per workload. An all-flash array might have no trouble accommodating cooler data. Still, if power efficiency is a priority, then that data might be better suited to higher-capacity HDD-based storage or archival tiers -- some of which can reduce power consumption by spinning down inactive drives.

This means that rather than placing an entire data set on high-performance storage, it may be more efficient to reserve the flash media for the data that is being actively used. Typically, only a fraction of the overall data is in use at any given moment, meaning that cooler data can be stored on slower but more energy-efficient tiers. And if some of the less frequently used data suddenly becomes important, it can be automatically moved to the high-performance tier, where it will remain until it "cools off."

Storage tiering can also sometimes reduce hardware acquisition costs by ensuring only frequently used data occupies expensive high-performance storage. It can also reduce power costs by keeping infrequently used data on lower-power, lower-cost storage that requires less energy and cooling. This, in turn, ensures that less energy is being consumed by storage, thereby making that excess energy available for use with AI workloads.

Balancing ESG Goals with Storage Needs

Organizations that design their storage architecture for operational efficiency might discover that their efforts naturally support ESG reporting, as reduced power consumption is objectively measurable and easy to document. In the case of an AI data center, however, the business justification for reducing power consumption might have less to do with meeting ESG goals and more to do with preventing a limited supply of power (or a limited power and cooling budget) from becoming an operational constraint.

In the AI era, those responsible for storage must look beyond performance and capacity planning. Today, storage decisions play a pivotal role in reducing an organization's TCO while also ensuring that limited data center power supplies can be used where they will deliver the greatest business value.

Brien Posey is a former 22-time Microsoft MVP and a commercial astronaut candidate. In his more than 30 years in IT, he has served as a lead network engineer for the U.S. Department of Defense and a network administrator for some of the largest insurance companies in America.