Enlighten Fusion Multiscale Network for Infrared and Visible Image Fusion in Dark Environments

Abstract

Most infrared and visible image fusion algorithms often struggle in dark environments where texture details in the visible image are largely obscured, although they are demon-strated to achieve good performance under normal illumination. To mitigate the dark environments issue, a novel Enlighten Fusion Multi-scale Network (EFMN) is proposed in this letter, which incorporates enhanced features at different scales into the main fusion network for lighting up the contexts in the darkness. With a sub-network enhancing the low-light visible image, multi-scale features are progressively enhanced and extracted. Then, a group of Fusion Modules (FM) are designed to fuse the features coarsely in multiple branches. Finally, the fused features are further refined by 1×1 convolution units to produce the resultant image. The processing of coarse fusion and then refinement at feature levels works effectively. Extensive experiments have shown that the proposed EFMN improves fusion performance in dark environments both subjectively and objectively. The improvements also facilitate typical down-stream vision tasks, such as object detection.