Fraunhofer Lighthouse Project NeurOSmart

As mobile robot systems become more autonomous, the number of sensors increases, the effort required to link their data increases, and with it the need for computing power to realize reliable and safe real-time operation. Architecture scalability, sufficient transmission bandwidth between sensor and data processing, and minimization of power requirements are the main challenges for the development of high-performance computers to be used in mobile systems. It is predicted that in less than 10 years, the required computing capacity in the sensor periphery will have to match that of a supercomputer today. This requirement can only be met by a combination of hardware and software components specifically developed for each other.

The NeurOSmart project aims to set a new standard for intelligent hybrid computing architectures in autonomous machines and transportation systems. For this purpose, a high-performance sensor system, AI-supported preprocessing and a novel high-performance, analog-neuromorphic, ultra-low-power in-memory accelerator chip are combined.

The prospect is an increase in energy efficiency of data processing by at least two orders of magnitude. This will enable the development of novel autonomous systems with previously unattainable intelligence and energy efficiency.

The NeurOSmart approach focuses on the direct integration of data processing intelligence into the sensor system. This reduces a significant portion of the computational load on the part of the HPC system in an environmentally and resource-friendly manner, so that the computational hardware in the sensor system can be adapted to its requirements directly during sensor development in the codesign.

As a pioneer of integration in a competitive sensor system, NeurOSmart uses an open scanning LiDAR system developed by Fraunhofer as a basis to provide direct access to the incoming data streams. In addition, a highly scalable, analog-neuromorph HPC chip is coupled with a sophisticated, AI-powered pre-processing pipeline to interpret the data directly at the sensor.

In total, NeurOSmart bundles the technical expertise of five Fraunhofer institutes, of which Fraunhofer ISIT is coordinating through Prof. Dr. Holger Kapels. For the participating Fraunhofer institutes, NeurOSmart opens up new, exciting opportunities to combine their respective technologies and thus to map the value chain of such a system from conceptual design to manufacturing and evaluation in an application-oriented environment.