Hear from:
The modulation-transfer function (MTF) is a well-established optical metric to quantify the ‚sharpness‘ of a camera system. We previously described a novel approach to use the MTF to quantify the quality of a simulation framework. Based on these results we now ask the question how well the MTF values correlate with the performance of evaluation algorithms, like traffic sign recognition or object detection. For this, we take a given database of automotive images (here: the Berkeley Deep Drive, BDD) and degrade these images with a parameterized optical model. We end up with several variations of the original database with different but realistic optical properties. Each simulated optical property is characterized using our MTF approach. Finally, a standard object-detection algorithm (car and person detection) is run on all database variations, showing the change in AI performance in dependence of the optical parameters. The results indicate that the MTF and the object-detection performance do not correlate well, as distinctly different MTF curves yield exactly the same precision-recall curves.