Edge AI startup reveals GenAI accelerator, $120M fundraise
The startup introduced a new GenAI accelerator for PCs and smart vehicles. The vendor's growth highlights the shift to training GenAI workloads from the cloud to the edge.
Edge AI chipmaker Hailo on Tuesday introduced its new Hailo-10 generative AI accelerators while revealing that it raised an additional $120 million in an extended Series C fundraising round.
The new funding round was led by investors including the Zisapel family, Automotive Equipment and Delek Motors. It brings Hailo's total funding to more than $340 million.
The Israeli vendor's new accelerator, or computation machine for AI workloads, enables users to employ generative AI on edge devices such as PCs, smart vehicles and commercial robots.
Hailo-10 can run the Meta Llama 2 7B large language model with up to 10 tokens per second, using less than 5 watts of power, according to the vendor. It can also process text-to-image model Stable Diffusion 2.1 at under 5 seconds per image, the vendor said.
Generative AI and edge devices
Hailo-10 and other generative AI edge accelerators come as the generative AI market has shifted to include edge devices.
While many generative AI workloads still need to be processed in the cloud and data centers, more training and inferencing of generative AI workloads are moving to edge devices such as laptops, virtual reality devices and smartphones.
That shift from cloud-based AI to edge AI has been very quick.
Olivier BlanchardAnalyst, Futurum Group
"That shift from cloud-based AI to edge AI has been very quick," Futurum Group analyst Olivier Blanchard said.
The shift creates a hybrid AI ecosystem with cloud computing at the top, on-premises in the middle and edge devices at the bottom, he added.
Generative AI at the edge is also aiding in the move from ideation to implementation of generative AI in enterprises, Gartner analyst Chirag Dekate said.
"At the end of the day, it is not just about what happens in the data center," Dekate said. "It's about how AI can be delivered everywhere."
Hailo-10's angle
For Hailo-10, the main benefit of running generative AI workloads on PCs, or even in smart vehicles, is power consumption, Blanchard said.
"The real advantage of this is going to be the lower power consumption, like the better performance per watt," Blanchard said.
Hailo-10 is capable of up to 40 tera operations per second (TOPS), which is faster and more energy-efficient than an integrated neural processing unit, according to Hailo. Neural processing units are specialized systems designed for machine learning algorithms.
The 40 TOPS performance level is in line with the Intel Core Ultra, an AI accelerator built for laptops and other edge devices, Blanchard added.
That Hailo-10 is mobile-friendly and can handle video inputs and process video data rapidly with low power is also important for smart vehicles such as electric vehicles, he continued.
"The more screens you have, the more processes you have -- all this stuff is a drain on the car battery," Blanchard said. "It's super important for vehicle-specific [offerings] not only to be super fast and super powerful, but also to consume as little energy as possible in the process of performing those tasks."
Hailo also uses a data flow-like architecture that simplifies how data flows through the processor and allows it to deliver more efficient compute, Dekate said.
Some limitations and the Israeli AI market
However, the problem with edge AI accelerators from Hailo and competitors such as Groq, Intel, AMD and Tenstorrent is that they lack standardization of the programming layer, Dekate continued.
Each company tends to have its programming model and stack, requiring a level of expertise from users. This differs from a cloud vendor such as Nvidia, which has a standard CUDA programming layer.
"Therein lies the tradeoff," Dekate said. "You will inevitably run into the problem in which you get better performance and better cost profiles, but programming complexity increases."
Another challenge is that the OEM channel partnership approach of vendors such as Hailo could hinder at-scale deployment.
"You will need to directly rely on a partnership with these companies," Dekate said. "These companies might not have the necessary ... personnel and business bandwidth to be able to address your customer requirements in this context."
Hailo is one of the many Israeli AI startups popping up and growing in the generative AI market, though the Israeli tech sector has been hurt by the war between Israel and Hamas in Gaza.
"Israel is taking a leadership role in developing new AI solutions," Blanchard said. "Not just on the software side, but also on the silicon side."
Another Israeli vendor, AI21 Labs, recently introduced Jamba, a new LLM that offers a 256,000-token context window.
Esther Ajao is a TechTarget Editorial news writer and podcast host covering artificial intelligence software and systems.
