The AI hardware and software provider is focusing on building massive data centers that do everything in the AI lifecycle, from data ingestion to inferencing and fine-tuning.
As a leading AI hardware vendor, Nvidia has long been a leading force in GPUs, far outdistancing competitors like Intel and AMD. In recent years, though, the AI vendor has begun to focus on AI factories.
An AI factory is a type of computing infrastructure -- really, a large-scale data center -- that manages the entire AI lifecycle. It requires a massive amount of storage, networking and computing resources. The AI factory handles everything in the AI workload, from data ingestion and training to fine-tuning and inference.
Nvidia and AI factories
Nvidia CEO Jensen Huang is a major proponent of the AI factory and has been working to include it as a significant part of the vendor's AI strategy.
Nvidia made a big stride toward that by acquiring Mellanox in 2019, a networking company for high-performance computing.
"It turns out that high-performance computing and AI are almost the same thing, and that's making a bunch of computers behave as a single computer," said Kevin Deierling, senior vice president of networking at Nvidia, on the latest Targeting AI podcast episode. Deierling was senior vice president of marketing at Mellanox before Nvidia bought it.
"The realization that I think Jensen had is that the computer is no longer a box that has a processor inside or even a GPU inside, but it's the entire data center," he continued. "These AI factories are indeed data centers with racks and racks and racks of GPU-based storage, but it's more than just a single computer with some cheap metal wrapped around it. We integrate within the rack. We've squeezed more and more of these GPUs together, and we scale them up."
It turns out that high-performance computing and AI are almost the same thing, and that's making a bunch of computers behave as a single computer.
Kevin DeierlingSenior vice president of networking, Nvidia
For enterprises, the key to using AI factories is scaling the use and fine-tuning of AI models.
"You want to do fine-tuning of a model, and you want to use technologies like retrieval-augmented generation inside of the enterprise. You're going to take advantage of that AI data factory building things at scale," Deierling said.
Some challenges
However, AI factories require a lot of energy. To reduce power demands, Nvidia is putting the GPUs in the factory closer and closer together. The vendor has also implemented liquid cooling.
Even with its attempt to manage the energy consumption of the AI factories, Nvidia still aims to do what it is known for.
"We have to continue to do what we've been doing, which is deliver higher performance per watt to keep this something that we can build and do sustainably," Deierling said.
Other than Nvidia, Dell is also a player in the AI factory market.
Esther Shittu is an Informa TechTarget news writer and podcast host covering AI software and systems. Shaun Sutner is senior news director for Informa TechTarget's information management team, driving coverage of AI, analytics and data management technologies, and big tech and federal regulation. Together, they host the Targeting AI podcast series.
