IEEE Transactions on Multimedia · 2022 · 422 citations · 58 references
ComputingComputer ScienceComputer VisionMachine LearningSensor FusionDeep LearningDigital Image ProcessingFeature-level FusionImage Feature EngineeringImage FusionRepresentation LearningConvolutional Neural NetworksDeep Learning-based FusionDeep Learning-based Image FusionImage Data FusionImage Feature FusionInfrared Image FusionSalient Feature ExtractionVisible Image Fusion
Infrared and visible image fusion is aims to generate a composite image that can simultaneously describe the salient target in the infrared image and texture details in the visible image of the same scene. Since deep learning (DL) exhibits great feature extraction ability in computer vision tasks, it has also been widely employed in handling infrared and visible image fusion issue. However, the existing DL-based methods generally extract complementary information from source images through convolutional operations, which results in limited preservation of global features. To this end, we propose a novel infrared and visible image fusion method, i.e., the Y-shape dynamic Transformer (YDTR). Specifically, a dynamic Transformer module (DTRM) is designed to acquire not only the local features but also the significant context information. Furthermore, the proposed network is devised in a Y-shape to comprehensively maintain the thermal radiation information from the infrared image and scene details from the visible image. Considering the specific information provided by the source images, we design a loss function that consists of two terms to improve fusion quality: a structural similarity (SSIM) term and a spatial frequency (SF) term. Extensive experiments on mainstream datasets illustrate that the proposed method outperforms both classical and state-of-the-art approaches in both qualitative and quantitative assessments. We further extend the YDTR to address other infrared and RGB-visible images and multi-focus images without fine-tuning, and the satisfactory fusion results demonstrate that the proposed method has good generalization capability.
58
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 76.3K citations · Full text
Image quality assessment: from error visibility to structural similarity
Zhou Wang, Alan C. Bovik, Hamid R. Sheikh et al. · IEEE Transactions on Image Processing · 2004 · 56K citations
Multiscale structural similarity for image quality assessment
Zhou Wang, Eero P. Simoncelli, Alan C. Bovik · 2004 · 5.9K citations