An extensively updated OpenShift AI will underpin RHEL AI and, if Red Hat has its way, enterprises' AI infrastructure from the edge to the public cloud.
DENVER -- Red Hat's OpenShift AI has its sights set on hybrid cloud infrastructure management for generative AI apps with a series of updates this week.
OpenShift AI, launched at Red Hat Summit last year, replaced the cloud-only OpenShift Data Science platform for MLOps with a hybrid cloud version that supports edge and on-premises infrastructure as well as distributed application management. It will also act as a base layer of infrastructure automation for the newly released RHEL AI product.
With OpenShift AI version 2.9 this week, Red Hat strengthened connections between generative AI frameworks and large language model (LLM) serving tools and the Kubernetes substrate and application development tools that form OpenShift AI. Now, customers can manage predictive AI and generative AI together, according to Steven Huels, vice president and general manager of Red Hat's AI business unit.
"OpenShift Data Science targeted predictive AI workloads and was a cloud service. ... We heard from a lot of our customers that they needed the ability to deploy on-prem and in disconnected environments and have more control over the systems themselves when we launched OpenShift AI," Huels said during a press briefing May 1. "That's when we also got into the generative AI space. ... What we put into OpenShift Data Science and the core [OpenShift AI] platform has been able to roll straight forward into the generative AI [version]."
The stage was set for the opening keynote at Red Hat Summit this week.
OpenShift AI 2.9 updates include the following:
Support for multiple model servers to run generative AI and predictive AI/machine learning apps together in the same OpenShift AI infrastructure. This feature is built on updates to the KServe open source custom resource definitions package of Knative, Istio and Kubernetes that supports container-based AI workloads. New integrations for KServe include support for vLLM, an open source library for LLM serving; Text Generation Inference Server, an IBM-led fork of the Hugging Face TGI toolkit for LLM inferencing; and Caikit, a set of Python AI application development tools.
Distributed AI app orchestration via the integration of KubeRay, a Kubernetes operator for the open source Ray Python AI distributed application framework, and CodeFlare, an OpenShift-specific software stack that handles queueing, resource quotas, management of batch jobs and on-demand cluster resource scaling for distributed applications.
Single-node OpenShift support for edge deployments of OpenShift AI, in technical preview.
An accelerator profiles feature in the OpenShift AI management interface for configuring hardware accelerators such as Intel's Gaudi 3 and AMD's Instinct. Red Hat will also integrate with Intel's Arc and AMD GPUs.
New partner tools integrations such as a certified OpenShift Operator for Run:ai's Kubernetes GPU scheduling utility; support for Elastic's Elasticsearch Relevance Engine vector search and transformer models for retrieval-augmented generation; and support for models from Stability AI.
Red Hat OpenShift AI targets hybrid cloud LLMOps
With ongoing turmoil over layoffs and licensing price hikes in the wake of VMware's acquisition by Broadcom, Red Hat has a unique opportunity to claim a lion's share of the hybrid cloud computing market, said Steven Dickens, an analyst at Futurum Group, in an interview with TechTarget Editorial this week.
OpenShift has a huge role to play, especially if you believe, as I do, that a vast majority of these apps will be deployed on premises.
Steven DickensAnalyst, Futurum Group
"Kubernetes and specifically OpenShift are a credible alternative to VMware's Cloud Foundation, especially if it's a greenfield deployment," he said. "OpenShift has a huge role to play, especially if you believe, as I do, that a vast majority of these apps will be deployed on premises."
The major cloud providers have made the first big waves in generative AI, but Red Hat has a potentially compelling value proposition in open source AI projects such as InstructLab, as well as infrastructure tools that accommodate edge and data center hardware alongside public cloud, said Torsten Volk, an analyst at Enterprise Management Associates, in an interview.
"It's the trivial infrastructure stuff [that matters] when you have 50, 60, 70 GB models that need massive GPU and storage space," Volk said. "It's incredible how quickly you can spend $10,000 a month in Google Cloud running something there."
There are still items on the to-do list for OpenShift AI, including bringing technical preview features to general availability later this year, and some gaps in its distributed AI application stack, such as machine learning feature stores, which Huels said will be added in a future release. Red Hat is also offering sneak peeks at the Lightspeed virtual assistant in private preview for OpenShift and RHEL here this week, but few technical details are available.
Red Hat must keep developing ways to bridge and consolidate workloads across multiple hybrid cloud and edge deployments and services, said Sid Nag, an analyst at Gartner.
For example, Red Hat continues to offer several separate tools for various aspects of cross-cloud network connectivity, including OpenShift Service Mesh, based on Istio; Red Hat Service Interconnect, based on the Skupper open source project; and Advanced Cluster Management, which integrates open source Submariner. This week, it added another: Red Hat Connectivity Link, based on the open source Kuadrant project. This is all not to mention the network automation tools it offers in Ansible Automation Platform.
"[Red Hat] needs to look at federating cross-cloud models for networking, data, security, AI, applications and business outcomes," Nag said. "Cross-cloud integration and federation will actually make multi-cloud adoption a reality."
It's also early for the Red Hat customer base in adoption of generative AI. Red Hat doesn't sell some AI features such as Ansible Lightspeed separately from its other Ansible products, so it's difficult to define exactly how many customers are using its AI tools in production so far, but it's a minority, Red Hat CTO Chris Wright said during a press briefing session this week.
"We have customers in production, [but] the number of customers in production with those capabilities isn't the same as a more traditional subscription from Red Hat, like RHEL or OpenShift," Wright said. "But there's a ton of interest in that area. So what we're looking for is creating the connection to the customer through our platform ... and seeing where we go, from early stage of prototypes for the concepts ... into [production]."
Beth Pariseau, senior news writer for TechTarget Editorial, is an award-winning veteran of IT journalism covering DevOps. Have a tip? Email her or reach out @PariseauTT.
Dig Deeper on Systems automation and orchestration
