DUDB: Deep Unfolding-Based Dual-Branch Feature Fusion Network for Pan-Sharpening Remote Sensing Images

Abstract

The proposed method aims to enhance the fusion of high-resolution multispectral (MS) images (HRMS) by extracting spatial and spectral features from panchromatic (PAN) images and MS images. However, existing pan-sharpening methods often suffer from the problem of missing spatial and spectral detail information. To better preserve these details, we introduce a dual-branch feature fusion pan-sharpening network based on deep unfolding. In this network, we utilize the algorithm unfolding iterative module (AUIF-Block) to continuously acquire detailed information from both MS and PAN images for image reconstruction. By leveraging the adaptive channel and spatial feature enhancement module (DEM-Block), the network can adjust spatial and channel features adaptively, leading to more accurate feature extraction and more complete image reconstruction. Finally, the detail-based fusion module (DBFM-Block) is employed to integrate and enrich the content of detailed information extracted from different channels, resulting in improved fusion performance. Experiments were conducted on QuickBird (QB) and WorldView-2 (WV2) datasets. Through qualitative analysis and quantitative comparisons, we demonstrate that this method outperforms existing approaches.