Interior monitoring can enhance safety, comfort, and convenience for all vehicle occupants. Today these systems can detect driver distraction, signs of drowsiness, and even whether a child has been left behind in the vehicle and provide alerts the driver to these critical situations. By using some of the same base signals that are used to detect the functions defined above, we may be able to effectively reduce other unsafe driving behaviors. This may include driving while under the influence of alcohol or perhaps driving a vehicle when being subject to sudden sickness. These interior monitoring systems will be critical for determining if a driver is able to perform the dynamic driving task.

Interior monitoring can enhance safety, comfort, and convenience for all vehicle occupants. Today these systems can detect driver distraction, signs of drowsiness, and even whether a child has been left behind in the vehicle and provide alerts the driver to these critical situations. By using some of the same base signals that are used to detect the functions defined above, we may be able to effectively reduce other unsafe driving behaviors. This may include driving while under the influence of alcohol or perhaps driving a vehicle when being subject to sudden sickness. These interior monitoring systems will be critical for determining if a driver is able to perform the dynamic driving task.

In order to minimize the injury of occupants during a crash, an intelligent adaptive occupant restraint system can use the information about body height and weight of each occupant to improve the safety of each one individually. Both information can be obtained with an interior camera that is normally used for driver monitoring tasks. The proposed method makes a first height prediction based on the vehicle’s occupant face only and fuse its result with further body height-related features. Subsequently, the weight estimation approach uses the height prediction and fuse this result with further body weight related features. This method embraces the utilization of convolution neural networks and a machine learning-based regression.

This session covers the latest techniques, challenges and opportunities in implementing InCabin monocular 3D sensing using 2D image sensors. Enabling a new category of use cases for enhanced safety and optimized comfort, this session will further cover how efficient inference on AI chips is capable of generating new types of depth-aware data and new monetization models inside this third living space

Current 2D IR DMS systems and upcoming next generation 2D RGB/ IR OMS systems (≈2026) are expected to rely fully on 2D information. This is a good first step for OEMs to integrate a camera inside the car to monitor head and eye state to alert the driver in case of drowsiness or lack of attention.

Sony believes that in the future, advanced occupant state, posture and context awareness will be needed to achieve a safer in-cabin for all occupants. Therefore several sensors need to be fused, to not only cover the monitoring of the driver, but understanding the activity and behavioral context for the driver and all passengers. With this understanding the active safety systems step up in performance and passive safety systems can be optimally supported, such as active restraint systems.

In the talk Sony describes the necessity of sensor fusion, confirms the statement by research and shows ways how to enable safe in-cabins

Today’s embedded sensor technology and location APIs make the car the perfect incubator for a new standard in infotainment. Trip Lab is developing responsive in-cabin experiences powered by Empathetic AI. Our Mission: to create digital health interventions that help drivers feel better when they step out of the car than when they got in. This presentation will showcase real use case examples for improved wellbeing in the car; driven by the ability to sense, understand and support any state of driving.

In-cabin applications are becoming more complex, offering more connectivity, monitoring, and more personalization. Driver and occupant monitoring applications are the main applications due to their integration in future regulations. Several technical developments have been observed related to 2D and 3D vision for driver monitoring. For occupant monitoring, radar technology can provide another solution for Tier-1s. But, in-cabin applications are not limited to these applications. Interior lighting is growing rapidly from functional lighting to a more personalized lighting and could even be used for communication and safety applications. Air quality monitoring is still in its infancy but PM2.5 sensors based on the light scattering principle start to be implemented by some OEMs like Volvo. In-cabin applications require more sensors, more computing and therefore the car architecture will be impacted. Moving from a distributed to a more centralized architecture with domain controllers will offer to OEMs the possibility to monetize software services.

Connectivity is increasingly being integrated and expanded into newer vehicle models to improve efficiency and safety for people and property.  This high level of connectivity poses increasing risks for external attacks on vehicle systems that automotive manufacturers must protect against.  Established cybersecurity methods are being leveraged and brought to automotive architectures to provide increased security.  New generation in-cabin sensors will need to support cybersecurity features to establish secure channel link between the sensor and host through mutual authentication and provide video stream integrity.  However, the level of protection required is not clearly defined.  The lack of a standard imposes a flexibility of cybersecurity features on the image sensor which we will discuss.

Driver monitoring systems (DMS) are crucial to driver safety. Traditional DMS systems use RGBIR sensors and AI models to detect driver states in real-time. However, these systems are sensitive to image illumination, contrast, and dynamic range, and could also raise privacy and security concerns due to the use of in-cabin cameras. In this session, we present an event-based DMS with dynamic vision sensors (DVS) to overcome the limitations above. We will discuss the properties of DVS and showcase how we integrate DVS into edge device along with developed event-based AI models, achieving robust DVS DMS with high privacy.

A family of IPs, named PREDICTS, able to precisely predict in real-time the sleep onset of a subject through the analysis of few selected physiological parameters have been developed over the years, in order to run both on contact (wearable) and contactless (RADAR) systems.

A detailed validation phase run in different incremental steps with a relevant number of subjects on overnight tests and on the dynamic vehicle simulator at AVL (Graz).

Since PREDICTS is “sensor agnostic” a scalable and flexible architecture has been defined in order to successfully address both after-market and OEM applications.

A pilot application, combining a camera-based system and a wearable device on a fleet of heavy-duty trucks in collaboration with Garmin, who is providing the sensing technology, is in progress. It is among the first validations in the transport/logistics field.

In this talk, we will present the key concepts and the preliminary results.

The InCabin revolution will expand from the initial focus on DMS to many vision-based applications, and the presentation will take a hands-on approach and discuss future InCabin vision applications through the lens of an actual lens manufacturer. The optical stack is often seen as a commodity nowadays, even though the lens is a crucial driver, differentiator, and innovator for performance. Applying practical optimization and tradeoff strategies on the lens level supports the development of a differentiated camera product that can satisfy regulations and automotive requirements on the one hand and balance costs on the other.

Vision-based AI is a dangerous approach for drowsiness detection. This presentation covers several risks and pitfalls automotive suppliers will encounter. These stem from misconceptions and incorrect assumptions about the nature of drowsiness. We present an alternative approach that is based on a physiological biomarker for drowsiness using eyelid movements. This methodology can easily be incorporated into all vision-based driver monitoring systems, today. In addition, we will discuss other neurological biomarkers that can be derived from eyelid movement, such as onset of Alzheimer’s, epilepsy and neurotoxicity.

Nowadays most drowsiness detection systems are based on visible drowsiness manifestations e.g., yawning, closing eyelids, etc. and therefore have a very limited time window to act. We present here a system based on breathing that detects drowsiness several minutes before any visible drowsiness manifestation appears. Breathing is captured through a 3D iToF sensor in the interior of the vehicle.

The sensor measures the movements in the Z-axis of the driver’s chest corresponding to the breathing signal. Thoracic Effort Drowsiness Detection algorithm (TEDD) analyzes this signal to predict several minutes in advance if the driver will become impaired to drive due to drowsiness.

The proposed DMS shows promising results regarding the breathing signal extraction. When compared to a chest band we obtain a Pearson’s Correlation Coefficient (PCC) of 0.98. Regarding drowsiness prediction we obtain the exact same results for both systems.

Today’s automotive technology is advancing at an unprecedented rate. User experience and satisfaction plays a critical role in automotive system development, especially regarding in-cabin system requirements. Due to this, vehicle imaging systems are moving away from the narrow field of view driver monitoring system towards a wider Field of View (FoV) to capture the entire cabin (occupant & back seat). This means combing the mandatory Driver Monitoring System (DMS) with more modern user experiences such as occupant identification and video surveillance, in one single sensor that reduces cost and impact on interior design.

In this session we will discuss how advanced freeform lens with smart pixel management maximise the resolution on the driver while capturing the entire cabin in varying lighting conditions. We will show how advanced image processing algorithms including dewarping deliver the best pixels for either human or computer vision to maximize the application efficiency.