A Dual-Mode Intensity and Polarized Imaging System for Assisting Autonomous Driving

Abstract

Multi-modal image fusion is beneficial to improve the target contrast. Taking advantage of the significant polarization characteristics of vehicles, the fusion images combining polarization and intensity characteristics will help the autonomous driving system to find and identify target objects with low contrast. To this end, in this paper, we propose a polarization-intensity image imaging system for autonomous driving, including hardware as well as algorithm part. The hardware part consists of a polarization camera, which is used to capture the DoLP and intensity images. The two kinds of images are fused by the proposed algorithm, named as HPFusion, to highlight the low contrast target. HPFusion is a hierarchical processing fusion network with multi-scale encoder-decoder architecture, in which different modules are used to extract features of different depths in the encoder subnetwork. Dense jump connections are used in decoder networks to take full advantage of the characteristics of different layers and scales. In the fusion module, the methods of addition and splicing are used to combine features of different scales from space and channel dimensions, and realize end-to-end image fusion at the same time. Our experiments on our own captured data sets show that our HPFusion can maintain high brightness and rich texture details of polarized objects for better scene representation and visual perception, in addition, our HPFusion also achieves better fusion performance, surpassing other advanced methods in qualitative and quantitative comparison methods. The vehicle detection results show that our system has a certain application prospect in autonomous driving.