Scaling AI data center cooling for high-density servers

Why air cooling still matters

For decades, air cooling has been the backbone of data center thermal management. For most existing data centers across North America operating at moderate power densities, air cooling remains a cost-effective and practical approach. However, the operational landscape is changing at an unprecedented pace, and the primary challenge for air cooling lies in the fundamental physics of heat transfer.

Air is inherently a poor medium for absorbing and transporting thermal energy compared to liquids. In fact, water has a heat capacity approximately four times greater than that of air on a per-mass basis. For example, removing 1 kW of heat with a 20 degrees Fahrenheit air temperature rise requires roughly 158 cubic feet per minute of airflow. If the allowable temperature rise is cut in half, the required airflow roughly doubles.

Even with highly optimized air cooling, the limit is around 30 kW to 40 kW per rack. Beyond this, the required air volume and velocity become impractical to manage. This results in unacceptable acoustic noise levels and operational instability.

Liquid cooling is closer to the heat source

Liquid cooling is becoming a practical requirement for many high-density deployments. The principle is simple: move heat capture as close to its source as possible. With a heat capacity over 1,000 times that of air by volume, liquid provides a viable path for managing the extreme thermal loads of current and future AI hardware.

Direct-to-chip (D2C) liquid cooling is a component-level strategy that targets the hottest elements within a server. The architecture replaces the air-cooling heat sinks on components with thermally conductive copper or aluminum cold plates. A sealed network of tubes circulates a liquid coolant that absorbs heat directly from the chip. D2C is often the first step for facilities introducing liquid into their white space.

Another major approach is the rear-door heat exchanger (RDHx), a rack-level solution that functions like a radiator mounted at the rear of the rack. An RDHx is designed to neutralize heat at the rack level before it can escape into the data center environment. The primary advantage of this method is that it allows operators to increase rack densities without re-engineering the entire room's cooling system.

Immersion cooling is one of the most efficient cooling strategies for high-density environments with rack loads above 100 kW. It submerges computing hardware in a thermally conductive but electrically non-conductive fluid. However, it requires purpose-built hardware, fluid-handling processes and significant facility modifications.

Scaling cooling depends on efficiency, monitoring and sustainability

Scaling cooling capacity in the high-density era is a long-term effort that extends beyond selecting a specific technology. It requires an integrated plan that includes modular design, intelligent monitoring and a commitment to sustainability.

A data center can deploy a scalable cooling strategy as needed, benefiting from its adaptability to varying server and power densities. This approach offers financial and operational flexibility while helping to avoid the expense of overprovisioning.

Methods such as RDHx can be retrofitted into existing data center cabinets. However, as power density increases, D2C liquid cooling becomes more suitable. In high-density deployments, immersion cooling can be implemented in self-contained, modular tanks. This approach ensures that cooling capacity scales with compute demand.

The cooling system must also be resilient. Fault tolerance and maintainability must be built into the system architecture. Components such as pumps, coolant distribution units and chillers can be deployed in N+1 or 2N redundant configurations to ensure continuous operation during a component failure.

A scalable cooling strategy must also include comprehensive monitoring and analytics. This data-driven approach requires a dense wireless sensor network. ASHRAE recommends six temperature sensors per rack. For liquid systems, sensors monitor flow rate, pressure and temperature within the coolant distribution units.

Data center managers can use this sensor data for predictive maintenance with machine learning algorithms. This enables proactive maintenance scheduling, reducing unplanned downtime and extending equipment lifespan.

Finally, sustainability has become a core business concern, forcing operators to balance power usage effectiveness (PUE) and water usage effectiveness (WUE). Advanced liquid cooling can reduce energy consumption, thereby lowering the PUE. However, operators must balance PUE against WUE as some highly energy-efficient cooling methods are water-intensive.

Final thoughts

The future of data center cooling will not be a complete transition from air to liquid, but a hybrid thermal architecture that applies the right cooling method to the right density profile. Optimized air cooling will remain essential across much of the installed base. In contrast, D2C liquid cooling, RDHx and immersion cooling will be deployed where rack-level heat loads demand them.

Cooling will become a planned dynamic infrastructure layer that depends on the scale of compute deployment. The data centers that succeed will be those that can convert available power into usable compute without creating thermal bottlenecks.