your123 -

AI vendor SambaNova intros next generation of DataScale

The AI vendor released an updated version of its DataScale system. The new system boasts improved performance and expanded memory for enterprises working with large models.

SambaNova Systems on Wednesday unveiled what the AI hardware and software vendor calls the next generation of its DataScale integrated hardware-software system.

DataScale lets enterprises train and deploy their own deep learning models.

SambaNova -- which competes with AI chipmaker and software vendor Nvidia and its DGX A100 systems built on the A100 Tensor GPU, among other AI vendors -- said its new DataScale SN30 system supports the largest AI models and can train GPT models at speeds as fast as any available. The SN30 is a step up from the vendor's older SN10 system.

Specialized and accelerated infrastructure

With the next generation of DataScale, SambaNova has improved the model performance of the system by doubling the processor speed and improving the memory capacity of the ecosystem, said Chirag Dekate, an analyst at Gartner.

Like other AI hardware vendors such as Nvidia, Intel and AMD, SambaNova Systems caters to enterprises that need more specialized infrastructure and accelerated production of complex models for applications in the banking, pharmaceutical and energy industries and scientific research.

SambaNova said the U.S. Department of Energy's Argonne National Laboratory is expected to test the updated platform in applications, including using GPT for gene generation, 3D convolution networks for neutrino physics, and prediction of tumor response to single and paired drugs.

"You are starting to see hardware vendors step up and develop capabilities in their processor architectures or accelerator architectures that can actually address larger models at faster timescales," Dekate said.

At the heart of the SN30 design is the capability to get faster compute when integrating hardware and software, said R. "Ray" Wang, analyst and founder at Constellation Research.

Software-defined approach

The DataScale SN30 system lets enterprises take a software-defined approach, Wang continued. "You can bounce from CPU to GPU based on the data types."

You are starting to see hardware vendors step up and develop capabilities in their processor architectures or accelerator architectures that can actually address larger models at faster timescales.
Chirag DekateAnalyst, Gartner

This is what differentiates the SN30 architecture from other AI production-level architectures on the market, Dekate said. Users can reconfigure their processors to meet the demands of their workloads, letting enterprises deliver high-efficiency performance for their neural networks.

"If you are an enterprise today, you no longer just have one processor you need to choose from. You can actually explore and evaluate multiple options," Wang said.

SambaNova also unveiled a new processor for its SN30-based Reconfigurable Dataflow Unit. While the vendor does not sell its chips separately, they are key components of its DataScale systems and Dataflow-as-a-Service, which provides customers pre-trained deep learning models, said Marshall Choy, senior vice president of product at SambaNova.

"This goes into the overall system architecture where we have built our systems to provide massive amounts of memory to run massive models," Choy said, referring to large language models like GPT that require large parameters, computer vision models that call for the highest resolution, and recommendation engines that need embedding tables.

"All of these requirements equate to the need for more physical memory in the system," he said.

Proof of concept

For enterprises that work with large deep learning models and foundation models like GPT, it's important to perform proof of concepts with vendors of GPUs and advanced processor architectures such as SambaNova, Dekate said.

"What you're likely going to discover is for larger, more complex architectures, technology architectures that have higher memory capacity and that are custom tuned for performance will likely deliver better value," he said.

While SambaNova did not reveal the pricing for DataScale SN30, it said it will now offer subscription pricing for DataScale and Dataflow-as-a-Service.

The vendor, based in Palo Alto, Calif., unveiled its updated platform on the second day of the AI Hardware Summit in Santa Clara, Calif.

Dig Deeper on AI infrastructure

Business Analytics
Data Management