When it comes to energy efficiency and sustainability in IT, rightsizing is critical. Optimizing the infrastructure model and the right hardware needed to run it should be the goal.
Research by TechTarget's Enterprise Strategy Group sheds light on the vital role that energy efficiency and sustainability issues now play in IT architectural planning and strategic decision making. Eighty-seven percent of organizations reported that an internal environmental, social, and governance (ESG) program accelerated their replacements of existing IT equipment, and 85% said they had eliminated a potential supplier due to ESG-related concerns.
With the rise of AI, especially generative AI (GenAI), the adoption of graphics processing unit (GPU) technology has increased. That adoption is, in turn, increasing the power demands of compute environments. Organizations must now stay hypersensitive to their energy consumption rates overall, including the amount of heat generated by servers and subsequent cooling requirements. Their end goal should be to make their own facility as energy efficient as a hyperscaler's.
Fortunately, many GenAI initiatives don't require supercomputers, and it is possible to rightsize GenAI models to suit particular use cases. The number of parameters involved, the number of concurrent users, and the type of data science required (i.e., fine-tuning versus retrieval-augmented generation) greatly influences infrastructure-related requirements. For example, some AI models can run on devices like regular desktop workstations, while other models would benefit more from running on high-performance Dell Precision workstations that have been engineered to handle large data sets efficiently.
It's a balancing act that involves assessing time and performance requirements against business expectations and costs, including power-related costs. AI model training typically requires more GPU computations and more time to complete. Conversely, machine learning inferencing (i.e., turning data points into a machine learning algorithm) requires low latency for query responses, with models stored in GPU memory.
Fortunately, Dell has been adding innovations to its PowerEdge servers -- including Dell OpenManage Enterprise Power Manager for GPUs, as well as air-cooling and liquid-cooling options -- to reduce the energy demands of a Dell compute infrastructure.
Dell focuses on a "components to the walls" approach to energy efficiency. Dell Modular Data Centers can deliver power usage effectiveness (PUE) rates that rival the hyperscalers. And Dell continues to develop more efficient infrastructure.
