Israeli chip startup Hailo has added three new vision AI chips designed for different use cases from the Hailo-8. While the Hailo-8 has mainly been used as a co-processor for vision processing in industrial edge boxes and aggregators so far, the Hailo-15 series is a system-on-chip (SoC) with accelerator, CPU and DSP on-chip, which will target IP cameras.

Hailo will continue to produce the Hailo-8 alongside the new product line.

“Many customers [are doing AI] sometimes on the camera, sometimes on the gateway, and ideally would like to reuse the software investment from one to another,” Hailo CEO Orr Danon told EE Times. “We’ve come up with a new lineup of products that specifically target smart cameras that rely on the same tech stack as the Hailo-8… [the stack] is new and improved, but fully compatible to allow deployment in various forms.”

While customers will typically buy either the Hailo-8 or the Hailo-15, there are some settings which might use both, Danon said—perhaps with vision processing at the endpoint and then aggregating streams from multiple cameras in a gateway.

CPU and DSP

The Hailo-15 series offers heterogenous compute, including a CPU for orchestration tasks, a DSP which provides flexible but more dedicated compute, and an AI accelerator to tackle the heavy lifting of AI inference.

“AI is not only the neural network, whether it’s convolutional neural networks or transformers, there are multiple aspects to it, and the DSP is an efficient tool to bridge the gap between the CPU and the AI accelerator,” Danon said. Both the CPU and DSP are licensed from third parties, and are not RISC-V-based, he added.

The Hailo-15 has less processing power than the Hailo-8, and its AI accelerator architecture is “similar to the Hailo-8, but improved,” Danon said.

The Hailo-15H offers 20 TOPS, Hailo-15M 11 TOPS and Hailo-15L 7 TOPS of INT8 performance (the Hailo-8 offers 26 TOPS). Power consumption depends on the workload, but customers are running below 2 W today, he added.

These levels of performance will suit smart cameras with multiple sensors, depending on how many tasks they want to run simultaneously, and the resolution, frame rate and prediction accuracy required.

“We have customers running over 20 tasks concurrently on this type of device… that’s heavy processing, but some customers [prefer] a more compact, cost-efficient solution with lower [performance] requirements,” he said.

Many Hailo customers will also use AI to enhance image quality, including low light performance, digital zoom and more—these applications are common today in smartphone cameras, but are yet to come to IP cameras.

Hailo-15 offers transformer support

Hailo’s software stack already supports transformer networks, including ViT for the Hailo-8, and this capability will also apply to the Hailo-15 series.

“We have customers who have already deployed transformers on devices,” Danon said. “[Transformers] are set to grow stronger, but I don’t think they will take over completely. They are very good at some tasks, but on some they’re not more efficient than CNNs. We also see some combinations, which are pretty powerful.”

Transformers’ key benefit is adding context to what is going on in images. Danon cites the classic example of an advertisement on the side of a bus— transformers can add context to CNN powered person detection, helping distinguish between a photograph of a person in an advert and a genuine pedestrian.

Another powerful combination of CNNs and transformers might be adding natural language description to images, he said.

As well as support for transformers, Hailo’s software stack has expanded support for different models via the company’s model zoo, and added a training environment for these models that enables productization.

While early samples are in customer hands, the Hailo-15 series will be sampling mid-year.

Architectural details of the Hailo-15 will be shared later this year, Danon said.

By: DocMemory
Copyright © 2023 CST, Inc. All Rights Reserved