We spoke to Paola Iacomussi, Senior Researcher at the Italian National Metrological Institute (INRiM) and lecturer at the University of Turin about one of the less-discussed challenges facing ADAS and autonomous vehicle development: how to build truly trustworthy perception systems.
Drawing on three decades of experience in metrology, road-environment characterisation and lighting applications, Paola explores why robust perception is not just about adding more scenarios or sensor modalities, but about understanding uncertainty, calibration and the real-world behaviour of sensors under changing environmental conditions.
Interveiw with: Paola Iacomussi Senior Researcher
1. What are the most persistent challenges in achieving robust perception across diverse environments, lighting conditions, and weather scenarios?
As a metrologist with 30 years of experience in the characterization of the road environment and lighting applications, my point of view is necessarily different from that of sensor and vehicle manufacturers. I’m more sensitive to trustworthiness in the metrological meaning that include data uncertainty and data comparison, with robust studies of metrological performances. So said, in my opinion the most persistent challenge is the absence of a fully metrological treatment (including the knowledge and impact of uncertainty on algorithms) of sensor performance and data in perception chains. ADAS/AV simulations often rely on synthetic sensor models, measured or simulated data, and equivalent environmental scenarios, but these inputs are commonly treated as nominal: sensor uncertainty, environmental-condition uncertainty, and uncertainty propagation into data fusion or ML algorithms are not explicitly considered. This is critical because sensor response changes with working conditions, weather, ageing, misalignment, accidents or maintenance operations.
Therefore, robust perception cannot be demonstrated only by testing performance in adverse scenarios, but requires calibrated sensors, uncertainty statements, traceability, also for adverse scenarios.
A second challenge is the oversimplification of the environment itself. In industrial simulations, road materials and visual targets are often represented by simplified models rather than by realistic spectral and geometrical properties. However, road surfaces, markings and signals change with lighting spectrum, shadows, glare, wetness, ageing and fading and are optimized for human driver perception and only in the visible range. This directly affects camera colour discrimination, object recognition and luminance-based perception, especially in adverse weather or complex lighting. From a metrological standpoint, the key issue is therefore to connect sensor performance, material characterization and scenario simulation through quantified uncertainty, expanding the radiometric range of interest (outside the visible range) instead of relying only on guaranteed nominal performance under selected test conditions.
2. How do current perception systems handle edge cases, and where do you see the biggest gaps in real-world performance?
Current perception systems usually handle edge cases by enlarging the number of test scenarios and perturbations in virtual or physical simulations. However, from a metrological point of view, the main limitation is that sensor models, sensor data and environmental conditions are often treated as nominal inputs, without propagating measurement uncertainty into the perception chain. This is particularly problematic in edge cases, because uncertainty impact can have more impact on perception and discrimination. For example an unusual lighting, sensor misalignment and/or maintenance effects may change the sensor uncertainty (namely the data output range) exactly when the algorithm has less redundancy and lower confidence.
The biggest real-world gap is therefore not only missing scenarios, but missing quantified trustworthiness of the data used in those scenarios. Edge cases should be assessed by linking sensor calibration, uncertainty statements, reference conditions and data fusion/ML behaviour. Otherwise, a system may appear robust in simulation while the real sensor output, under degraded or borderline conditions, has an uncertainty large enough to affect object detection, lane recognition, traffic-light recognition or emergency braking decisions. This is why uncertainty impact on ML and AI is identified as a necessary step for reliable ADAS/AV performance.
3. How should engineers approach validation and testing of perception systems to ensure reliability at scale?
Engineers should move from a mainly performance-driven validation approach to a metrological validation chain. At present, the impact of measurement uncertainty on data fusion and ML/AI algorithms is only beginning to be addressed and uncertainty propagation through the perception chain is still not mature enough to claim truly robust and reliable models at scale. A key point is that calibration is not the same as testing or alignment check.
Many current ADAS procedures check or realign sensors, but metrological trustworthiness requires a relation between reference quantities, sensor indications and associated uncertainties. For mass production, however, individual calibration of millions of sensors is not realistic. This is why large-scale statistical calibration approaches are being investigated, together with traceable calibration services. The IMU/MEMS case shows that statistically grounded metrological characterization can already evaluate sensitivity, dispersion, stability and uncertainty under different working conditions, providing a model for scalable sensor trustworthiness.
Don’t miss Paola’s expert tutorial ‘Why Your Perception System Depends on 50-Year-Old Road Standards’ on Tuesday 9th June, at 3:30pm EDT in Room 140 C/D exclusively for our Learn Pass holders!
Interested in In-Cabin monitoring technology?
With a pass to AutoSens USA, you’ll also get full access to our co-located sister event, InCabin. Take a look at the full agenda for InCabin here >>
