How to gain value from underutilized data center capacity
Data center capacity often goes underused due to fragmented visibility. Understanding resource needs and optimizing workloads can enhance efficiency and reclaim valuable capacity.
Identifying underused capacity does not provide data center operators with guidance on how to use it. The answer depends on why the capacity is sitting unused.
Two facilities can show similar utilization levels for completely different reasons. One could have resources reserved beyond what its applications need, while another could have enough compute, but a workload that cannot use it efficiently.
Siraj Aziz, analyst for cloud and data center at Omdia, a division of Informa TechTarget, pointed to workload optimization, predictive automation and better visibility as ways organizations can improve capacity use. But those approaches only help when the underlying capacity can be recovered.
For infrastructure leaders, the question is where capacity can be put to productive use with the infrastructure already in place, and where capacity requires a larger investment or a different use.
Find out why the capacity is underused
Aziz said one challenge is that infrastructure visibility remains fragmented.
Infrastructure teams use different systems to monitor networks, AI workloads, cloud resources and applications. However, physical infrastructure capacity is tracked separately. The individual systems show what is happening within their own layer without explaining why the specific infrastructure space remains underused.
For instance, a platform team can see an idle CPU. At the same time, an application is unable to scale further, or facilities teams can know that a row has reached its power limit even though a capacity dashboard shows empty rack space.
The first step is to connect those discrete views to the workload. Operators need to understand what resources the application requires, what it consumes and what prevents additional work from being placed. This includes compute, memory, storage and networking, as well as available rack power and cooling, all of which affect placement.
Low utilization can then be broken down into different problems. Capacity can be overprovisioned, poorly allocated or limited by the application itself. For example, a physical constraint, such as limited space, can prevent additional deployment. The available infrastructure can also be technically functional but unsuitable for the target workload.
Operators also must distinguish between unused capacity and capacity kept in reserve. Some infrastructure is intentionally left available for failover, maintenance or predictable workload peaks. If the capacity is reclaimed, it can reduce resilience in the target application.
The objective is to identify which resources can be recovered without creating another constraint.
Application architecture can make the distinction harder. Aziz noted that horizontal pod autoscaling depends on the application being sized and designed so that additional replicas can increase useful work. Predefined hardware profiles can also restrict changes to resource allocation and power states.
The objective is to identify which resources can be recovered without creating another constraint.
Optimize and rightsize existing capacity
Once operators understand why capacity is underused, they can determine how much additional workload the existing infrastructure can support.
Aziz described predictive automation, proactive scaling and workload optimization as important characteristics organizations must have to manage capacity effectively. Operators can adjust capacity as workload demand changes and identify resources that have been allocated beyond what the application requires.
Rightsizing is one way to do that, in which a workload can reserve more CPU than it consumes, leaving those resources unavailable to other applications even when the hardware is lightly used. Adjusting those allocations can free up capacity, but the sizing decision must consider workload peaks, latency requirements and failover conditions.
Autoscaling extends the same idea to changing demand. Horizontal pod autoscaling can add or reduce application replicas, while cluster scaling can change the worker capacity available to those applications. But scaling creates additional useful capacity when the application can scale with it.
The location of resources can also strain capacity. For example, a cluster can have sufficient total CPU or accelerator capacity but still be unable to place another workload because resources are spread across different hosts. Operators must schedule and consolidate to restore those resources to a usable state.
The cheapest source of new capacity can sometimes be capacity that the organization has already paid for.
Google found that automated GPU sharing increased aggregate GPU throughput by 50% and enabled the evaluated cluster to support 19% more GPU jobs. A GPU assigned to one workload can have execution time available for another compatible workload.
The cheapest source of new capacity can sometimes be capacity that the organization has already paid for.
Decide whether to modernize, repurpose or walk away
Optimization has limits. Capacity that remains underused after rightsizing, scheduling and other operational changes may require a different decision, as the existing facility no longer matches the workload.
Bruce Bateman, chief analyst for semiconductors at Omdia, argued that this decision should begin with demand. When discussing an operator that already has unused capacity, he summarized the issue as "It's demand, it's not compute." Bateman said that adding newer infrastructure does not create the workload demand. The operator must determine who will consume the additional capacity and what that workload requires.
The next critical question is how much of the existing facility must change to support it.
In the case of a single constraint, operators can justify a targeted upgrade. But the economics change when one upgrade triggers several others. Bateman explained that newer compute can require additional storage, higher-capacity routers and internet connectivity. Higher-density systems will introduce new power and cooling requirements.
Leaders need to calculate the complete investment required to make the target workload usable. If a compute upgrade requires modifications to electrical distribution, cooling, networking, storage and building systems, the operator must decide whether to convert the facility to a different workload class.
Bateman said that workload conversion is not always necessary. He added that a facility already supporting enterprise applications, databases, hosting or private cloud workloads can continue serving those customers if the economics remain attractive. Older infrastructure does not lose its value because it cannot support the highest-density AI systems.
But in supporting the boom in AI systems, Bateman said inference is one possibility for operators who are not positioned to support large training environments. Private AI, regional inference and other local workloads can fit some facilities where customers value connectivity, locality, privacy and control.
The decision is to determine which workload offers the strongest business case for the existing infrastructure and how much additional investment that workload deserves.
Building inference capacity because the broader AI market is growing creates the same risk as any other speculative upgrade.
Bateman said that the broader idea is to "stay in your lane." In this context, the phrase does not suggest avoiding modernization, but points to understanding what the facility can support economically before trying to turn it into something fundamentally different.
The decision is to determine which workload offers the strongest business case for the existing infrastructure and how much additional investment that workload deserves.
Abhishek Jadhav is a technology journalist covering AI infrastructure, semiconductors and advanced computing systems.
