Can edge computing make AI more sustainable?

Understanding edge AI's sustainability advantage

Edge AI offers a series of potential sustainability advantages, including the following:

Reduced data transmission energy

Routing raw data to a centralized cloud data center consumes energy at every network hop. Edge architectures reduce that load, as they process locally and send only events, exceptions or summaries upstream rather than continuous raw data streams.

In applications such as industrial monitoring or video surveillance, where devices continuously generate high data volumes, reduced energy can also lower network energy costs and cloud compute requirements.

Localized processing efficiency

Cloud GPU infrastructure is generally optimized for training at scale. Applying it to repetitive, targeted inference tasks means running at a fraction of its designed capacity while still carrying the full overhead cost. In contrast, edge hardware is designed for low-power inference.

"If you deploy overpowered hardware at every site, run models continuously, underutilize the equipment and do not account for hardware lifecycle, the energy story gets weaker fast," said James Sheridan, CEO of Sheridan Technologies, which designs and builds embedded systems and AI-enabled products.

Lower cooling and infrastructure overhead

Cooling is one of the biggest power draws for AI data centers. Cooling accounts for up to 30% of data center energy consumption, according to the International Energy Agency's April 2026 analysis.

Higher GPU densities drive more heat per rack, pushing facilities toward more intensive cooling infrastructure. Edge devices require neither active cooling systems nor water-cooled infrastructure.

When does edge AI make sense?

Edge AI is not a universal fix for all AI deployments, but it does make sense in specific situations.

"I would start with the workload, not the architecture preference," Sheridan said.

Where edge computing works best

Edge performs well for latency-dependent and always-available applications, in connectivity-limited or data-sensitive environments where cloud transmission creates compliance risk. Manufacturing quality control, autonomous systems and point-of-care diagnostics are common examples. For continuous high-volume inference, upfront hardware costs are typically offset by lower ongoing cloud, transmission and energy spend.

Not every organization has the option to use cloud, though.

"The question should not be: 'Is edge better than cloud?' The question should be: 'What environment does this system actually need to survive in?'" said Tyler Saltsman, CEO of EdgeRunner AI, who works across defense and mission-critical infrastructure deployments.

Despite those advantages, edge deployments also have drawbacks. Edge AI runs on neural processing units (NPUs), purpose-built chips with no dominant market standard, often requiring models to be re-engineered for each chip variant.

"The NPU ecosystem is deeply fragmented right now; models need hand-tuning per chip variant, which kills the ROI case for most IT departments that don't have specialist ML [machine learning] engineering teams," said David Viney, CIO at Alchemy Consulting and an AI governance consultant who works across enterprise organizations.

Companies like Rentokil and General Electric's rail division have made it work, Viney said. Both built IoT infrastructure years before AI adoption accelerated and have the engineering depth to manage model updates, patching, drift monitoring and hardware replacement at scale.

Where cloud works best

Cloud works best for large-scale model training and general AI workloads that require significant compute. Applications with frequent model updates and situations requiring centralized data aggregation all remain better suited to cloud infrastructure.

There is also a sustainability counterargument to edge that IT leaders should consider.

"Cloud AI providers are already dealing with power, cooling, utilization and infrastructure efficiency at scale," Sheridan said. "If their infrastructure is much more optimized than yours, moving the workload local does not automatically improve ROI or sustainability."

Sustainability metrics for edge AI

IT leaders building or evaluating edge AI programs should track the following metrics as part of any green AI strategy:

• Power usage effectiveness. PUE is the ratio of total facility energy consumption to IT equipment energy use. It's useful for benchmarking centralized infrastructure but has limited direct applicability to edge deployments, where there is no facility overhead to measure. The relevant comparison for edge is energy consumed per inference task, not facility-level efficiency.
• Energy consumption per inference. This is a more effective metric for edge than PUE. It measures the joules or watt-hours per AI transaction, across edge and cloud options for the same workload. A site running edge inference on a workload previously handled in the cloud can produce a direct before-and-after comparison.
• Carbon footprint per AI workload. Total greenhouse gas emissions attributable to a defined AI workflow, including electricity source, hardware efficiency and network transmission energy.
• Total cost of ownership, including energy. This includes hardware acquisition, operational overhead, energy costs and model management across the full deployment lifecycle.
• Compliance and ESG reporting, including Scope 1, 2 and 3 emissions. Scope 1 covers direct emissions from owned or controlled sources. Scope 2 covers purchased electricity. Scope 3 extends to supply chain emissions, including cloud vendor infrastructure, hardware manufacturing and device replacement cycles. In the U.S., the Securities and Exchange Commission (SEC)'s climate disclosure rule requires public companies to report on Scope 1 and 2 emissions. In the EU, the Corporate Sustainability Reporting Directive (CSRD) and AI Act set additional energy reporting requirements for AI systems.

The future of edge computing, AI and sustainability

Two developments are reshaping edge AI infrastructure.

The first is small language models (SLMs). SLMs are compact task-specific AI models designed to run on lower-power hardware and, as such, are well-suited for edge AI use cases.

The second major trend is agentic AI, which is creating new demand for persistent local compute and inference.

The model that makes most sense routes routine work to local hardware and reserves cloud capacity for tasks that genuinely need it, Sheridan said. SLMs have improved enough to make that split practical.

"More systems will use a hybrid pattern, small local models for fast, private, repetitive or event-driven work, and larger cloud models only when deeper reasoning or broader context is needed," Sheridan said.

Agentic AI is where the efficiency case gets harder. Agents running continuously and calling models on every cycle can consume as much compute on local hardware as they would in the cloud.

"The sustainability benefit depends on discipline: Use smaller models, run them only when needed, cache results, escalate selectively and measure the actual workload," Sheridan said.

The shift toward agentic AI will pull infrastructure investment toward distributed colocation environments closer to existing enterprise systems, rather than expanding hyperscale capacity, Viney said. In his view, that pattern is inherently more efficient than routing everything to a large, centralized facility.

Compliance requirements are making workload-level energy tracking a business necessity. The EU's CSRD and AI Act require energy and carbon disclosure for AI systems. In the U.S., SEC climate disclosure rules require public companies to report Scope 1 and 2 emissions.

Those pressures might change the question organizations ask about AI, Sheridan said, from "Can we use AI here?" to "What model, running where, at what cost, with what energy profile and what business value?"

Viney sees the same shift coming and is blunt about where most organizations stand.

"Organizations that are already tracking carbon per workload will have a compliance head start. Most aren't," Viney said.

Sean Michael Kerner is an IT consultant, technology enthusiast and tinkerer. He has pulled Token Ring, configured NetWare and been known to compile his own Linux kernel. He consults with industry and media organizations on technology issues.