A pass is required to view this video.

Vehicle Occupant Heart Rate and Respiration Rate Estimation Based on a RGB-NIR Camera

Sign in | Register to Bookmark

Hear from:

Patrick Laufer
Development engineer,

IAV

Released on July 04, 2023

Modern vehicles are now fitted with interior cameras that can sense both visual and NIR modalities. Given that an interior camera is already installed in vehicles for driver monitoring purposes to prevent accidents, there is an opportunity to use them for monitoring vital signs of occupants. The proposed presentation will provide the latest research in the field of estimating heart rate and respiration rate with a single RGB-NIR camera using deep learning and the optical flow approach. We chose the face as the region of interest for estimate the heart rate because it is typically uncovered, allowing the camera to capture subtle variations in color and brightness caused by changes in blood volume due to arterial pulsations. To estimate the respiration rate, we used an optical flow algorithm that recognizes chest movements within the video frames and maps them to a respiratory frequency. Additionally, we tested our approach not only in a laboratory environment but also inside a vehicle with different test subjects. The presentation will include a statistical evaluation of our proposed method. The driver’s vital signs can predict or indicate critical events, such as sudden heart attacks, strokes, or fatigue, in the early stages. This has the potential to enable a controlled stop of the vehicle before occurring. Another potential use case for in-cabin vital sign estimation is in telemedicine, where a vehicle can be transformed into a mobile medical space. A remote doctor can access the interior camera and vital parameters of the patient to diagnose an illness and adjust medication accordingly. This presentation will discuss whether a single RGB-NIR camera is reliable enough to estimate the heart rate and respiration rate inside a vehicle based on our findings. Additionally, we will identify the potential problem cases associated with this approach.

Want to watch this session?

You must be logged in and have a pass to watch this session.
Scroll to Top