Every major Kubernetes distribution supports NVIDIA hardware, yet despite that, the AI silicon giant decided that there was still something missing.
This week during the KubeCon North America conference in Salt Lake City, the AI infrastructure giant open sourced NVIDIA Kubernetes Engine (NVKE) and NVIDIA Kubernetes Recipes (NVKR), meant to help operators run managed Kubernetes services on GPU infrastructure.
The gap NVIDIA is targeting is the cluster work around the hardware, while eliminating potential handoffs between server hardware and platform management teams. Platform operators must provision bare metal machines, confirm node identity, upgrade clusters and replace failed nodes. NVKE automates that lifecycle. NVKR builds and signs the software that NVKE deploys.
While there might appear to be some overlap with existing capabilities from vendors such as Red Hat OpenShift and SUSE Rancher, NVIDIA officials say it isn't meant to be a competitor.
"NVKE isn't meant to replace OpenShift or Rancher," said Warren Barkley, vice president of AI factories at NVIDIA. "NVKE and NVKR are an open source reference stack for AI factories. Reference architectures have traditionally been documents, but we saw an opportunity to go further by building a reference that operators and partners can deploy, test, and build on."
How NVKE and NVKR work
Barkley said that NVKE is meant to help platform operators offer managed Kubernetes on GPU clusters for internal users, while NVKR gives operators validated recipes to build their own signed, version-pinned images.
"NVKE is one piece of a larger stack that's open by design, and operators run the service and own their customer relationships," Barkley said.
NVKE pulls machine images through a standard Kubernetes interface.
"NVKE natively uses the Kubernetes Cluster API to provision infrastructure through a standard interface," Barkley explained. "We built and open-sourced a Cluster API provider for NICo, our bare-metal infrastructure controller for GPUs."
The goal of the effort is to accelerate deployments so workloads can start running faster. Barkley said that NVIDIA's focus is optimizing software and infrastructure together. That work accelerates the time it takes a cloud provider or platform operator to go from installed hardware to an AI factory serving tokens to users. He noted that it also improves efficiency and utilization once workloads are running.
"NVKE … requests GPU nodes directly from NICo, so new and replacement capacity goes from the rack into a customer's cluster without a hand-off between teams," Barkley said.
The NVIDIA advantage here is that the company builds both the GPUs and the software that runs on them. As such, Barkley noted that a validated Kubernetes stack can be ready for customers on the day a new GPU generation ships.
Vendors see a partner, not a competitor
The major Kubernetes distribution vendors, including Red Hat and SUSE, view NVKE and NVKR as complementary to their efforts.
"We are collaborating with NVIDIA to determine how best to leverage NVKE and NVKR alongside OpenShift and our broader Red Hat AI platform," Joe Fernandes, vice president and general manager of the AI business unit at Red Hat, told TechTarget. "We see ongoing ecosystem investment in upstream Kubernetes tooling as complementary to advancing enterprise AI adoption overall."
SUSE also calls NVKE complementary. It plans to extend its management tools to NVKE clusters where customers want that.
"SUSE builds one Kubernetes, but manages all, and where customers want to bring NVKE-provisioned clusters into the wider estate, we will extend that management through SUSE AI Factory and SUSE Rancher Prime," Rhys Oxenham, vice president and general manager for AI at SUSE, told TechTarget.
NVIDIA calls the projects a foundation for partners. Eight partners, including Mirantis, Rafay and vCluster, have said they intend to adopt them.
In terms of how the technology will be integrated by partners, Barkley explained that some will adopt the complete stack. Others already have mature components, so they will keep those and add NVIDIA components built for GPU infrastructure, such as validated cluster configuration recipes, telemetry pipelines, and GPU fault remediation.
"A third group will borrow our patterns to improve their own offerings, which is one reason we released NVKE and NVKR as open source software," he said.
Analysts see a complement…mostly
IDC analyst Arnal Dayaratna also expects NVKE to complement established Kubernetes platforms such as OpenShift and Rancher Prime. In his view, those platforms give enterprises a consistent way to deploy applications and enforce policies across clusters.
"NVKE sits closer to the hardware and automates how operators provision, update, and maintain Kubernetes clusters on bare-metal GPU servers," Dayaratna said. "I expect Kubernetes vendors to bundle NVKE or integrate parts of it into their offerings so customers can get that automation through the platforms they already use."
The NVKE project page on GitHub states that it is not a Kubernetes distribution. But one analyst still places it in that category, for a narrow set of workloads.
"It is an integrated software stack play, it won't unseat existing distributions on the traditional use cases but will be the #1 distribution for Nvidia centric workloads," said Holger Mueller, principal analyst and vice president at Constellation Research. "That is what it is built for - not to compete with generic, all purpose Kubernetes distros."
Sean Michael Kerner is an IT consultant, technology enthusiast and tinkerer. He has pulled Token Ring, configured NetWare and been known to compile his own Linux kernel. He consults with industry and media organizations on technology issues.
