Autonomous driving is still in its infancy, and has not even reached the consumer market. In order to do so, it must be made safer and more reliable and far less expensive. This will require many sensors on the car, producing massive amounts of data. The cost of these sensors, but also of processing the massive amounts of data must be greatly reduced.

In the community, the debate on the relative merits of centralized, zonal and decentralized compute architectures has not been settled. In the near future, a centralized processing and fusion architecture seem to fit best with some important OEM and TIER-1 roadmaps. However, some of the best current stand-alone “smart” sensors rely on embedded processing, taking advantage of easier certification, faster reaction times and higher reliability.

Centralized architectures are attractive because the separation of compute and sensing resources offers more choice in selecting sensors, more flexibility in reallocating computational and memory resources between sensors. A single compute platform also avoids the potential problems of needing to support different sensor specific platforms.

However, centralized compute platforms have several important downsides that will render them less and less attractive when levels of automation increase. They constitute single points of failure and will need to be duplicated as backup systems, just as essential sensors and control systems in airplanes are. They concentrate all heat production in a single box, limiting the number of computations. They rely on high-speed links, adding to cost. The zonal architecture is a better compromise in this respect.

My personal prediction is that architectures will become more and more decentralized over time, but before this happens, consensus will need to grow on what types of data to exchange between sensors and compute units, and in what level of detail. Irrespective of the chosen architecture, the energy demands and the cost of solutions built with current hardware and software need to be improved by a large factor for consumer autonomous driving to be realistic at all.

The cheapest computations are those that are never made at all. Active sensing strategies can therefore be part of the solution: carefully adapting frame rates, processing only well-chosen parts of the data or – even better – not sensing all parts of the scene with all sensors all the time. Perhaps this too will be addressed at a future AutoSens.

Meanwhile, one way to reduce computations is context-adaptive processing: instead of relying on large neural networks handling all possible combinations of scenes, weather and light conditions… we can use a smaller network, designed only to handle standard weather and light conditions. The clever bit is to add a smaller auxiliary neural network to adapt the main network’s output to compensate for weather and light influences. This calibrates the “almost final” output of the neural network for the current sensing conditions. This not only saves computation, but also requires less training.

Improvements in neural networks (e.g., power driven architecture search) are another part of the solution. On the hardware level, novel processing paradigms, e.g., neuromorphic neural networks promise lower power consumption and ultrafast response times at the lowest levels of processing.

No matter what processing hardware is selected, neural networks require massive amounts of information transfer, internally in high-end compute chips. Chiplets and “networks on chips” promise to provide power-efficient, massively parallel internal data transfer. Chiplets also enable a new, modular approach to customizing high-end Application Specific Integrated Circuits to specific automotive needs. This solves another major problem: the prohibitively expensive Non Recurring Engineering cost of their design.

In summary, the data processing challenges of automotive sensor fusion will be addressed at the hardware and software level. AutoSens features presentations and a panel discussion on future hardware processing architectures and on design methodologies for low power neural networks. Other presentations will address new fusion methods to avoid needless computations, thus providing complementary solutions.