The company is integrating its robotics components into NVIDIA’s Holoscan Sensor Bridge (HSB) and expanding support for high-fidelity simulation within NVIDIA’s Isaac robotics ecosystem.

The collaboration brings ST’s sensors, actuators and microcontrollers into the reference component set compatible with NVIDIA’s HSB, which is designed to standardise and synchronise data across robotic platforms. High-fidelity digital models of ST technologies are also being introduced into Isaac Sim to help developers close the gap between simulation and real-world performance.

Initial integrations available now include Leopard Imaging’s depth camera, enabled by ST technologies and now compatible with the HSB, as well as a high-accuracy model of an ST inertial measurement unit (IMU) within Isaac Sim.

Rino Peruzzi, Executive Vice President for Sales & Marketing, Americas & Global Key Account Organisation at STMicroelectronics, said the company’s strong presence in the robotics community underpins the partnership. “Our collaboration with NVIDIA aims to unleash the next wave of cutting-edge robotics innovation with developer and customer experience streamlined at every step, from the inception of AI algorithms to the seamless integration of sensors and actuators,” he said. “This will accelerate the evolution of sophisticated AI-driven physical platforms.”

Deepu Talla, NVIDIA’s Vice President of Robotics and Edge AI, said high-fidelity simulation and integrated hardware support are essential for advancing autonomous systems. “The integration of STMicroelectronics’ sensor and actuator technologies with NVIDIA Isaac Sim, Holoscan Sensor Bridge and Jetson platforms provides developers with a unified foundation to build, simulate and deploy physical AI at scale,” he said.

The NVIDIA Holoscan Sensor Bridge allows developers to unify and streamline data collection from multiple ST sensors and actuators - a key requirement for building reliable Isaac Sim models and reducing the “sim-to-real” performance gap.

ST and NVIDIA’s joint goal is to simplify the use of ST technologies with Jetson platforms, providing pre-integrated solutions that combine STM32 microcontrollers, imaging technologies, depth sensors, time-of-flight (ToF) devices and motor-control systems. This approach is particularly aimed at developers working on humanoid robot designs.

One example is Leopard Imaging’s stereo depth camera, which uses ST imaging, depth and motion-sensing technologies. It is expected to support a wide range of designs across robotics manufacturers, research institutions and industrial robotics teams.

Developers working on advanced robotics face significant challenges, including high costs, complex modelling requirements and the need for large datasets to train AI systems. High-fidelity simulations require substantial computing resources, and selecting the right variables to randomise during training demands deep domain expertise. Poor decisions during this process can create unrealistic scenarios or slow down model convergence.

To address this, ST and NVIDIA are working to deliver hardware-calibrated models that reflect the real-world behaviour of ST’s components. Following the release of the IMU model, ST plans to introduce simulation-ready versions of its ToF sensors, actuators and other integrated circuits. These models are built using benchmark data captured from actual ST hardware and are optimised for the Isaac Sim environment. The NVIDIA HSB is also being incorporated into ST’s toolchain.

The companies expect the availability of more accurate component models to improve simulation realism, reduce training cycles and lower development costs for next-generation humanoid and autonomous robotic systems.

By: DocMemory
Copyright © 2023 CST, Inc. All Rights Reserved