What if “better-looking” data doesn’t mean better performance? Ahead of AutoSens USA 2025, Prof. Valentina Donzella challenges the traditional view of image quality, showing why perception-optimised data—designed for machines, not humans—is the new benchmark.
Deep Neural Networks (DNNs) and Artificial Intelligence (AI) will be deployed in Intelligent Vehicles to accomplish several tasks, from de-noising to perception, from tracking to prediction and control. In particular, given all the noise factors that can affect the quality of the data produced by the different perception sensor technologies in automotive [1-3], there is a focus in the community in developing the ‘best’ de-noise and enhancement techniques to improve sensor data quality before deploying any perception algorithms [4]. To give an example, in imaging, ISP (Image Sensor Processing) and image enhancement algorithms are used to produce ‘good-looking’ images…but ‘good-looking’ for the human eye. Are these processes really optimised for DNNs in automotive? Are they able to produce the best ‘images’ for DNN-based perception?
For example, Fig. 1a) shows a frame captured in night time condition with its segmentation ground truth below. Fig. 1b) shows the same frame enhanced via the URetinex low light enhancement (LIE) network with below the segmentation on the enhanced frame. It is clear that some of the details of the semantic content have been lost, however the image is ‘better looking’ and with more details for the human eye. Finally, Fig.1c) shows a perception-driven LIE proposed by Wei et al., where the enhancement of the dark frame is guided by the perception (in this case segmentation) performance. The enhanced frame is less pleasant to the human eye and with a green tinge, however the perception performance is better than when evaluating frames enhanced by current state-of-the-art image enhancement techniques, including URetinex.
Noise Factors and X-i-L testing
The Intelligent Vehicles – Sensors group, headed by Prof. Valentina Donzella, has published 3 papers breaking down and classifying into 5 categories (i.e. piece to piece, usage, interaction, environmental, change over time) the noise factors affecting 3 of the main perception sensor technologies used for Assisted and Automated Driving (AAD), namely LiDAR, camera, and 4D-RADAR, with respectively 16, 30, and 23 noise factors identified. As a part of the Sim4CAMSens project (https://sim4camsens.org/) and in collaboration with the AutoSens Brussels community, industry and academic experts have tried to classify which of the identified noise factors need to be tackled with the highest priority in order to ensure AAD functions. Amongst all the different types of noise factors, noise due to environmental conditions, such as luminosity, precipitation, wind, etc. has been recognised as the most critical one, as it might significantly affect data quality for the different sensor modalities, but also it is highly unpredictable, difficult to reproduce and to mitigate. Projects like the EU Horizon ROADVIEW (https://roadview-project.eu/) have been focused specifically on tackling robustness for assisted and automated systems in harsh weather conditions. However, some of the identified ‘high priority’ noise factors are very difficult to reproduce or to test for in a real automotive environment, see Table 1. This challenge highlights the need for good sensor models, allowing for noise injection and robustness testing in X-i-L (anything-in-the-loop), from full simulation to vehicle-in-the-loop testing. An example of an implementation of noise injection on perception sensors in a vehicle-in-the-loop setup is shown in Fig.2.
Table 1 – Summary of the Noise Factors rated as ‘high priority’ since they are highly variable, difficult to predict and reproduce
(Violet = Difficult to test in controlled condition, Green = Can be partially tested in controlled conditions, Black = Noise factor not applicable to sensor)
Noise Modelling and Data enhancement/restoration
Noise modelling for perception sensor and the generation of realistic and accurate ‘noisy’ data have become a focal investigation point in the Perception Sensors Community, and key discussion topic at events like AutoSens and InCabin. Noise modelling enables several key activities, to name a few:
- testing of filtering, de-noising and data enhancement algorithms;
- testing the robustness of Deep Neural Networks to different types of noise;
- testing of Perception and Fusion techniques;
- developing better (more robust) sensor technologies;
- creating pairs of ‘ideal’-‘noisy’ data, for benchmarking, DNN training, etc.
- X-i-L testing with noise injection real-time.
Some examples of physics-based camera and LiDAR noise models developed by the Intelligent Vehicles – Sensors group as a part of the ROADVIEW are shown in Fig. 3.
However, one of the widespread assumptions is that ‘better data’ will produce better perception results. This assumption is particularly true for cameras, where several de-noising (e.g. de-raining, de-snowing, etc.), filtering, low-light enhancement algorithms have been generated with the aim of creating or re-generating images that are nice-looking to the human eye. These images are traditionally assessed with Image Quality Metrics, that in most of the cases have been proposed to evaluate if images are satisfactory for the human visual system. Fig.1 shows how better image enhancement (for the human eye) might not be the better enhancement for DNN consumption.
Fig 3: Left to right, camera top, LiDAR bottom: real frame and then the same frame with modelled fog characterised by 30 m and 10 m of visibility. Figures from ROADVIEW work package 3 [7].
Perception driven data enhancement
The Intelligent Vehicles – Sensors group has been proposing something radically different from the process of restoring data to generate ‘filtered’ or ‘enhanced’ data that is close to the original one or close to what is pleasant for the human users. The proposal is articulated into 3 main research directions, discussed below.
- New Quantitative, Qualitative evaluation metrics need to be proposed and correlated with the perception quality. It is not always true that better IQA metrics support better perception, and this is discussed in [8]. For example, the widely used PSNR correlates very poorly with DNN based vehicle object detection. This is similarly true for pointclouds [9]. It is therefore important to use traditional metrics and also propose new metrics and correlate them with a wide variety of DNN perception tasks and their performance. Metrics should be versatile, e.g. allowing to measure the quality of different colour spaces, Bayer and raw images, etc.
- A new approach to Machine Learning Tasks. The great majority of deep neural networks have been designed to work with de-noised RGB (Red-Green-Blue) images. In turns, RGB images are produced for human fruition and cause an unnecessary increase of data (3 colour channels instead of one) and modification of the pristine detected data. It has been demonstrated that current DNNs can work with Bayer data, with state-of-the-art results [10]. It is possible that computer vision tasks and DNNs for specific usage in AAD will be created and optimised just to consume Bayer data, improving current solutions.
- A synergetic approach in DNNs, balancing enhancement tasks and perception. An example is shown in Fig. 4, giving a schematic view of the proposed OTTO-Net framework. The input frames are ‘corrected’ through two stages, which are guided by the perception performance (e.g. detection and segmentation). As shown in the Figure, the processed frame does not maximise Image Quality Analysis (IQA) Metrics, however it has improved IQAs and luminosity with respect to the initial frame, reduced noise, but particularly it has the highest perception accuracy. This result is enabled by a tailored design of the network and its loss functions, but given the impressive results, it can certainly change completely how we approach data enhancement tasks.
Wrap up
The community of Sensors’ experts in the automotive field has seen remarkable changes and achievements in the last few years. There are still challenges ahead, particularly to ensure that vehicles equipped with AAD functions can drive safely; most of these challenges are linked to the resilience and robustness of the sensor suite. I personally look forward to seeing more process on perception driven sensing, and to sharing any interesting findings in this field from my research group!
Perfect data isn’t what it used to be. Catch Prof. Donzella at AutoSens USA 2025. Her insights offer a critical shift in how we develop, test, and deploy perception systems for real-world driving.
Interested in in-cabin monitoring technology?
With a pass to AutoSens Europe, you’ll also get full access to our co-located sister event, InCabin. See the key topics for InCabin here >>
Check out the highlights from AutoSens USA 2024 ⬇
References
[1] P. H. Chan, G. Dhadyalla and V. Donzella, “A Framework to Analyze Noise Factors of Automotive Perception Sensors,” in IEEE Sensors Letters, vol. 4, no. 6, pp. 1-4, June 2020, doi: 10.1109/LSENS.2020.2996428.
[2] B. Li, P. H. Chan, G. Baris, M. D. Higgins and V. Donzella, “Analysis of Automotive Camera Sensor Noise Factors and Impact on Object Detection,” in IEEE Sensors Journal, vol. 22, no. 22, pp. 22210-22219, 15 Nov.15, 2022, doi: 10.1109/JSEN.2022.3211406.
[3] P. H. Chan, S. S. Roudposhti, X. Ye and V. Donzella, “A noise analysis of 4D RADAR: robust sensing for automotive?,” in IEEE Sensors Journal, doi: 10.1109/JSEN.2025.3556518.
[4] A. Mohammed Raisuddin, I. Gouigah and E. E. Aksoy, “3D-UnOutDet: A Fast and Efficient Unsupervised Snow Removal Algorithm for 3D LiDAR Point Clouds.” Authorea Preprints, 2024.
[5] Z. Wei, V. Donzella, L. Wang, “Better Low-light Image Enhancement, Better Low-light Perception for Self-Driving, ” submitted IEEE T-IP
[6] P. H. Chan, J. Robinson, D. Gummadi, Y. Poledna and V. Donzella, “D3.4 Library of validated physics-based parameterised noise models,” ROADVIEW, 2024.
[7] https://roadview-project.eu/
[8] D. Gummadi, P. H. Chan, H. Wang and V. Donzella, “Correlating traditional image quality metrics and DNN-based object detection: a case study with compressed camera data.” in Authorea Preprints, 2023.
[9] V. Donzella, P. H. Chan, D. Gummadi, A. Mohammed Raisuddin and E. E. Aksoy, “LIDAR De-Snow Score (DSS): combining quality and perception metrics for optimised data filtering.” Authorea Preprints, 2024.
[10] P. H. Chan, C. Wei, A. Huggett and V. Donzella, “Raw Camera Data Object Detectors: An Optimisation for Automotive Video Processing and Transmission,” in IEEE Access, vol. 13, pp. 21695-21706, 2025, doi: 10.1109/ACCESS.2025.3529287.
