We present a unified perception approach integrating data from a pair of stereo LWIR cameras and a LIDAR. It allows the integration of a set of functionalities from thermal IR raw data to a 2.5D representation. Thus, the contribution of IR camera is evaluated through:
– Several dedicated light-weight CNN-based detectors for pedCETrian and multiclass objects on LWIR images,
– The assessment of the stereo LWIR through point clouds comparison with LIDAR with a specific focus on a custom stereoIR camera rig platform,
– Fusion of 3D data from IR stereovision and semantic data from IR inference into a 2.5D occupancy grid with clustering and object dynamics analysis,
In addition, a dedicated multi-sensor platform provides a reference baseline to address the issues of calibration and computational load balancing on embedded targets for the implemented functionalities.