Engineers working on embedding AI in edge devices who are just now doing their first machine learning project have high hurdles to overcome, but recent developments in the industry may offer encouragement.

“The education, I think, has gotten better in the last couple of years,” said Eta Compute CEO Evan Petridis. “There’s been a lot of efforts by the vendors and by others on the education side.”

Petridis, whose company is building a silicon-agnostic tool chain for edge AI, said the reality is that AI is tough and super-fast moving, and those who haven’t kept up with recent developments are at a disadvantage. There’s also a different mindset compared with other engineering disciplines, he told EE Times during a recent panel.

“I think culturally, or by training, embedded systems people think deterministically,” Petridis said. “And I come from a traditional engineering background, so I think deterministically. And you know, the AI world is rooted in data science, and it’s a probabilistic world.”

The panel discussion, “Is Software Evolving Quickly Enough for the TinyML Tsunami?” was part of EE Times’ AI Everywhere 2023 virtual event in November, which is available for streaming here. Even though the panel’s title focuses on software, panelists talked about hardware, silicon, skills, field support and life cycle management too.

TinyML is a type of machine learning (ML) that allows models to run on smaller, less-powerful devices on the edge. It’s deployed in applications ranging from continuously monitoring machines for predictive maintenance purposes to smart home appliances.

Making up for lack of knowledge

Marc Dupaquier, managing director of artificial intelligence solutions at STMicroelectronics (STMicro), summed up the issue by citing data from seven years ago: “I remember a study from IBC that about only 1% of the embedded developers had some kind of AI skill. Probably 3 or 4% now, but still, it’s very low.”

In addition to the AI/ML knowledge and software hurdles to overcome, hardware resources in embedded devices are constrained, said Deepak Mital, senior director of architecture at Synaptics, a total solutions provider for AI.

“You’re constrained by the memory,” he said. “You’re constrained by the connectivity. And all of a sudden, you have been put into a small tunnel, and you have to run as fast as you can.”

The challenges include using different frameworks, getting the right quantization, and getting the right accuracy, he said.

“Last but not least, and very, very important, security is very, very critical,” said Mital. “When a lot of these engineers, they look at applications, they many times don’t consider security, and in the end, the customer goes, ‘Hey, what about my security? I have this IoT device that’s running my automation line. Tell me how am I not exposed here’.”

It’s up to industry

Blumind AI CEO Roger Levinson, whose company focuses on all-analog, extremely low-power, energy-efficient solutions for neural networks and AI systems, said it’s up to the industry to simplify the process for customers.

“At Blumind, our approach is [to] eliminate the need to have software in the system to run inference. Focus on the data on the job at hand. Train the model, and then you’re done. And it just runs on the software incredibly efficiently, and you won’t have made a bunch of hardware bottlenecks.”

Another potentially helpful solution is autoML, the process of automating ML tasks.

STMicro recognized the lack of AI knowledge and skills in embedded engineers, which led to its decision to have an autoML application.

“The goal was, let’s try to address the processing piece for people who have no AI skill, don’t want to have AI skill, and want to get to resolution without the complexity of it,” he said.

“It’s time for that field to come out of research and be used right now,” he said.

The alternative is to have a custom model, which means investing in a software tool chain. To overcome that dilemma, Synaptics is crafting model-building software, which he thinks will be critical for tinyML devices.

Problems yet to be solved

Also very critical for tinyML edge devices is life cycle management, Dupaquier said.

He sees STMicro customers who are just now in the early stages of production, and starting to anticipate questions about their model, such as, “How do I update it in the future? How do I refresh it?”

Thankfully, the experts at companies like STMicro aren’t starting from “zero.” The learning curve was larger and steeper for traditional AI, he said. And the knowledge gained from what he terms the “big world” of traditional AI can be applied to tinyML, he said.

Another issue yet to be solved is providing field support, said Mital.

By: DocMemory
Copyright © 2023 CST, Inc. All Rights Reserved