For many enterprises, AI is no longer an experiment; it's now part of everyday operations. According to Deloitte's 2026 State of AI report, worker access to AI increased by 50% in 2025. Yet, AI's continuing growth creates new concerns about the technology's environmental impacts—in particular the vast amounts of electricity and water needed to power and cool AI infrastructure. One report, by United Nations University, projected that AI data centers will consume 945 terawatt-hours of electricity by 2030.
It's up to CIOs to balance scaling AI responsibly while still achieving sustainability goals to create an operational strategy that supports growth. To do that, sustainability must be a core operating metric.
The CIO's bind: AI growth vs. sustainability commitments
Companies are scrambling to manage the demand AI places on enterprise resources.
"Most organizations are running AI far less efficiently than they think," said Arun Sahu, head of AI, data and applied intelligence for alliant. "Oversized models on simple tasks, redundant calls because no one caches, retrieval over messy data that forces three attempts where one would do, [and] employees pasting the same prompt 15 times because they were never trained on the first one."
Inference – the process of using a trained model to generate outputs or decisions – puts additional strain on infrastructure and resources. When performed at scale, inference request volume soars--potentially driving higher energy consumption and resource use.
Infrastructure and grid constraints, such as power availability and data center capacity, can throttle AI growth. As a result, CIOs must consider energy availability versus projected consumption when planning AI infrastructure and capacity requirements.
"Compute demand can grow in months, while expanding power generation, grid capacity and data center infrastructure can take years," said Anant Adya, executive vice president and head of delivery – Americas, at Infosys. "CIOs therefore need to think about AI, energy and infrastructure as one strategic system rather than three separate issues."
Cloud bills and infrastructure budgets only illustrate part of the picture; they may not account for environmental costs such as electricity consumption, water use or embodied emissions.
"The most effective AI strategies measure more than model performance; they evaluate the full infrastructure footprint required to support AI over time," said Dr. Sthitie Bom, vice president of global data factory IT at Seagate Technology. "Our approach is built around a continuously evolving value framework that connects infrastructure decisions to cost, business value and sustainability, allowing us to assess power, cooling, space, utilization and equipment lifecycles as AI workloads scale."
AI sustainability is a cross-functional effort. CIOs should work with CFOs and sustainability leaders to align competing priorities, guide technology decisions and establish KPIs, such as:
- Energy, both measured and estimated.
- Carbon emissions.
- Water consumption.
- Total cost and cost efficiency.
- Energy and emissions intensity.
- Business value and business outcomes.
Where AI's environmental impact lives
Scaling AI programs depends on understanding where AI causes environmental impact across its lifecycle. CIOs should consider several major areas of AI's environmental impact when making decisions about AI infrastructure, workflows and usage.
Data center energy, power, and cooling
Data centers and supporting AI infrastructure consume significant amounts of energy, especially as AI scales to larger models and increased usage. As AI becomes more commonplace, even more power and resources will be needed to support AI workloads and infrastructure, including graphics processing units, central processing units, networking and storage. These resources include cooling systems to manage heat and ensure equipment operates reliably.
Water consumption and local resource pressure
Water-based cooling systems require energy, and some use substantial amounts of water. Electricity generation may also include indirect water consumption depending on the source. The extent of water-related impacts depends on several factors, including cooling technology, energy sources, local climate and infrastructure design, which makes total resource use more complex.
Training versus inference
AI training and inference consume resources in different ways. Training requires substantial computing resources over a defined period. By contrast, inference creates ongoing computing demand and usage may be more unpredictable. Agentic AI and multi-step workflows further increase demand through repeated model calls.
Embodied carbon
Embodied carbon—emissions generated as part of the hardware lifecycle that spans manufacturing, installation and end-of-life processing--is an essential component of AI's environmental footprint, but it can be harder to track than other factors.
When AI scales, hardware demands also increase. Leaders may want frequent upgrades and replacements to create the most efficient AI systems, but this can also increase embodied emissions.
A practical framework to balance sustainability and AI
Understanding where AI creates environmental impact is only half the battle. CIOs must consider sustainability goals and use them to guide decision-making about how AI is used. It's up to CIOs to create a strategic, practical framework to manage AI's environmental footprint without sacrificing growth. CIOs should focus on several best practices for practical implementation, including:
- Choose the right models. Matching models to business needs ensures that the AI model matches the capacity of the task, rather than a larger model that may be underutilized. "Most production workloads don't need a frontier model," said Sahu at alliant. Knowledge distillation – which transfers knowledge from a larger model to a smaller one – and pruning helps reduce computational demands and resource consumption while maintaining the accuracy and performance required for the intended use case.
- Bake efficiency into applications. Identifying high-cost use cases like unnecessary model calls, overly long prompts and repetitive processing helps optimize AI processes and control consumption. Caching results and setting limits and usage guardrails also helps reduce unnecessary computation, AI energy consumption and associated costs. "A huge share of enterprise AI is the same question, asked again. Every repeat is energy spent twice for an answer you already had," said Sahu.
- Build smart infrastructure. Be intentional when building AI systems. This improves infrastructure efficiency and reduces unnecessary resource consumption. Compare energy availability, cooling requirements and infrastructure capacity when considering cloud-based or on-premises systems. "In many environments, infrastructure demand can grow faster than the number of AI applications, especially when workloads require repeated access, replication, retention, or specialized performance characteristics," said Seagate's Bom.
- Make sustainability part of governance and operations. Sustainability should extend to governance and operations, including integrating sustainability with machine learning operations and financial operations, controlling shadow AI usage and ensuring service-level agreements include sustainability and compliance guardrails. "Environmental sustainability should similarly be treated as a design consideration rather than a separate reporting exercise after an AI system has already been deployed," said Adya at InfoSys.
Alison Roller is a freelance writer with experience in tech, HR and marketing.
