Visual perception in today’s ADAS focuses on two separate domains, i.e., surround sensing and driver monitoring. Their progress over the last decade has offered great promise in increasing safety on our roadways. We present a novel system that aims to understand the complete driving situation based on the visual perception of the surrounding traffic scene and driver’s gaze. By modeling the driver’s past and present attention state, and tracking the surrounding object’s present and future movement, not only can we identify what the driver is looking at, but we can further infer the driver’s awareness of the surrounding object’s intention, allowing the system to achieve a comprehensive understanding of the situation.
The ADAS system, the driver, and the traffic environment are the three elements in the L2/L2+ settings. ADAS system must not only perceive the environment and the driver separately, but it should also understand the interplay between the driver and the environment. The situational understanding system we present addresses this objective and can infer the driver’s awareness of the traffic scene and their intention therein.