Data center energy efficiency a must to meet ESG goals

Sustainability impact of data center expansion

The ESG effects of massive data center expansion are clear:

• Energy consumption. Data centers consumed about 415 TWh (terawatt hours) or 1.5% of global electricity in 2024, according to a report from the International Energy Agency (IEA). By 2030, this is expected to double to 945 TWh and 3% of global electricity consumption. 
• Water usage. Data centers are using water -- lots of it – to cool the immense levels of heat generated by AI computing. This is only going to increase as more data centers are built. West Water Research projects that data center water usage will grow by 170% by 2030.
• Carbon emissions. Renewable energy sources have made their way into the data center, but fossil fuels remain the predominant power source. Embodied carbon in construction materials and hardware also contribute to data center carbon footprints.
• E-waste. The frequent hardware refresh cycles used for AI training are significant generators of electronic waste.
• Land use. There are several ways that data center build-outs affect the physical environments around them. This includes loss of wildlife habitats, farmland, wetlands and green spaces, as well as noise from constantly operating ceiling fans. 

The enterprise data center moves home

Most of the attention around data centers and the environmental impacts have focused on the massive facilities that are being built by the cloud AI giants, including Meta, Google, AWS and Microsoft. But enterprise AI development is largely an on-premises data center story. CIOs must decide how to bring AI fully into the enterprise data center in new or retrofitted facilities or in co-location or hybrid scenarios.

For now, most of a CIO's AI spend is going to cloud service providers and to AI model builders such as Anthropic and OpenAI, according to Alex Cordovil research director at Dell'Oro Group, a telecommunications and data center infrastructure market research firm.

However, Cordovil expects enterprises to start bringing more AI infrastructure on-premises, under their own roof. This is both a strategy to control cloud costs and to enable more customization of AI models toward their own needs.

"Some large companies have already begun that journey, but we do not expect it to become a mainstream trend later in the decade," he said.

There are many reasons why enterprises are prioritizing their own data centers, according to Scott Sinclair industry analyst at Omdia, a division of Informa TechTarget.

One of this is data sovereignty, in part because of U.S. and European regulations, but also because AI often uses private data, including PII, trade secrets and source codes in models that organizations want to keep in their own data centers, Sinclair said.

"Many software companies have a rule that all of the core code for their software products must stay in their own data centers and not put on the public cloud or any cloud," he said. "So, if they're going to run AI to help developers augment their code, it has to run on-premises because that's where their code lives."

Another major factor, which also enters the sustainability conversation, is the overall demand on processing, GPUs, memory, data storage and networking. This infrastructure demand for AI is very costly to run on public cloud infrastructure, Sinclair said.

It's also expensive to run AI in an enterprise's own data center, but there are elements that allow it to run more efficiently, so it can cost relatively less than the public cloud, he said.

"It also gives you more control over capping those costs, because if it's running in your own infrastructure, you're not going to get a surprise bill from your own data center because you've already bought all the stuff," Sinclair said.

Kill the zombie servers

While there's no silver bullet for CIOs to address the ESG issues resulting from AI-driven data energy consumption, there are ways to mitigate the problems and be more environmentally responsible.

For an organization to become more energy efficient in its data center, it first must determine how much energy it uses and how much it will need, according to Tony Harvey, vice president and analyst at Gartner.

The first step to this is gathering data from the organization's electricity bills, he said. This allows the CIO to know how much power they're using and break down where the power is going – compute, cooling and ancillary applications.

"Because then you can start to work out your efficiency," Harvey said. "Your perfect efficiency would be that 100% of power is going to the compute, which is impossible because you're generating heat that you're going to have to get rid of."

Once an organization has accurate data on the amount of power it uses, the CIO can begin to plan improvements. One of the major power drains are zombie servers that hiding in most enterprises, Harvey said.

For example, there are a lot of VMware servers running now and as many organizations are now planning to migrate from VMware, they need to inventory many WMs they have and who owns them.

"Then they find out that they have a thousand zombie VMs that somebody turned on, and nobody ever turned off," he said.

The best way to consider power consumption and running AI is to not necessarily buy new and more powerful servers, but to make existing servers operate more efficiently, Harvey said.

"The more efficient you are, the more efficient you are using the hardware that you've already purchased," he said. "This means that you're not wasting the company's money on hardware that's not being used and the power on hardware that's not actually being used."

To liquid cool or not

Data center design is key to the efficiency that results in improved energy usage. But this gets into another ESG issue – the use of water to cool the heat produced in data centers.

There are two parts to the design that need to be considered, Harvey said. One is removing heat from servers, and the other is ejecting that heat from the data center to the outside world, which requires liquid cooling.

There are tradeoffs that must go into the data center cooling design. One way is to use airflow through a chiller that works like a refrigerator, which has effectively zero water usage but uses more power. The other is to use water, which is extremely efficient at removing heat and uses less power, but can consume vast amounts of water.

"There's always going to be some power usage and water usage," Harvey said. "But the more efficient you are, the less power you use, the less water you use."

The optimal way to think about data center energy consumption is to maximize efficiency, he said. Two simple concepts are the foundation for this – not overcooling the data center and getting rid of all the idle servers.

"Just run what you need to run, and if it's not being used, turn it off," Harvey said. "It's exactly the same advice as what you should do in your home."

The trade-off between air cooling versus liquid cooling is a vital element in the energy usage issue, agreed Sinclair.

There's a lot of innovation and excitement around the concept of liquid cooling, but the data centers that enterprises currently use are not set up to do liquid cooling, he said.

"If you're building a new data center today, you would likely optimize liquid cooling, but building a net-new data center is not in the picture for many traditional enterprises," Sinclair said. "But if you have a data center that was designed for air cooling, it's often cost-prohibitive to try to retrofit that to do liquid cooling."

This means that most CIOs are looking at air-cooled data center designs and major server vendors including Dell and HPE are developing air-cooled innovations.

CIOs need to focus on efficiency as much as possible both to minimize the impact on the environment and to minimize costs for the company, he said.

"Often the power envelope that's available to enterprises is a fixed thing," Sinclair said. "Most companies have a limit to what they can do, so they're trying to maximize the value they can create from that envelope, which means you need to be more efficient."

CIOs also need to consider ESG implications as they decide between retrofitting an existing data center or building new ones for the increased AI demands, Cordovil said.

Retrofitting uses the embedded carbon of existing infrastructure, which doesn't add Scope 3 emissions for the bricks and pipes used in a new facility, he said. However, in most cases a retrofitted facility will run less efficiently than a purpose-built one.

"The calculation here is non-trivial, and company leadership will need to weigh several different factors to best inform the decision," Cordovil said.

Budgeting for power

As enterprises move more into the AI era, they need to start thinking about power budgets in a similar way to their overall monetary budget, Sinclair said. They only have so much power, so they need to maximize that power "spend."

"Enterprises are already thinking in those terms, but if they're not, they need to start thinking about infrastructure investments in that framework," he said. "Because the demand for AI is not going to subside anytime soon. The more successful you are with AI, the more you want to do more of it, which drives more interest."

However, the ability of enterprises to improve efficiency in data center mechanical systems is likely not enough to offset the amount of power that AI will draw, said Cordovil.

"Cooling systems account for only a fraction of total energy consumption and even if we make them 10 or 20 percent more efficient, that will not offset the power draw of the accelerators," he said.

There is some promise in better chip efficiency and other promising early-cycle technologies, but the power demand holds the cards.

"The reality is that unless new chip architectures deliver efficiencies of more than an order of magnitude, that additional capacity gets consumed quickly by pent-up demand," Cordovil said. "It's good old Jevons paradox at play."

Jim O'Donnell is a news director for TechTarget, where he covers IT strategy and enterprise ESG.