Effective Use of Long-wave Infrared Technology for the Development and Optimization of Deep Learning based Intelligent Thermal vision Systems for Smart Vehicular Applications
Deep learning and Computer Vision have shown immense strength in real-world applications. This presentation will focus on using thermal imaging technology for designing an intelligent forward sensing system that should be effective in all weather and environmental conditions. This work is carried out under Heliaus project funded by European Union’s Horizon 2020 research and innovation programme and France, Germany, Ireland, and Italy. The systems work by deploying the thermally tuned deep learning networks on GPU & single-board EDGE-GPU computing platforms for onboard automotive sensor suite testing. The state-of-the-art object detection models are trained and fine-tuned on a large-scale novel C3I thermal dataset comprising of more than 35K distinct thermal frames collected from 640×480 uncooled thermal cameras along with 4 different large-scale publicly available thermal datasets. The trained network variant of the YOLO object detector is further optimized using SoA neural inference accelerator (TensorRT) to explicitly boost the frames per second rate and cut the overall inference time. The optimized network engine increases the frames per second rate by 3.5 times when testing on low-power edge devices thus achieving 11 fps on Nvidia Jetson Nano and 60 fps on Nvidia Xavier NX GPU-Edge computing boards.