Testing of an ML (Machine Learning) and AI-based perception system shall be realized as a part of the safety assurance of ADAS (Advanced Driver-Assistance Systems) and ADS (Autonomous Driving Systems). A safety-aware perception system to be validated needs the knowledge of an ideal result. This ideal result, a validation dataset, shall contain all the valuable information about the surrounding traffic and environment the perception system should perceive. The validation dataset is fundamental to the reliability and integrity of a perception system. A perception system relies not only on the quality of the validation dataset but also on its integrity. Therefore we need to address it as a part of the perception system safety case.
The presentation will focus on the elements that can expose insufficiencies in the dataset and provide sufficient argumentation on the safety aspects a validation dataset must fulfill to validate a perception system at different stages.