CNN-based fusion and classification of SAR and Optical data

Abstract

Image fusion combines the images of different spectral, spatial, multi-date, as well as radiometric data to achieve a better quality image for improved classification results. Recently, Convolution Neural Network (CNN)-based classification algorithms are extensively used for remote sensing applications. Keeping this in view, present work proposes to use CNN-based fusion and classification of Sentinel 1 (VV and VH polarization) and Sentinel 2 datasets acquired over an agricultural area near Hisar (India). For image fusion, three CNN-based approaches are used to fuse Sentinel 2 (10 m) data with VV and VH bands of Sentinel 1 data. After fusion, classification was performed using 2D-CNN classifier to judge the performance of fused images in terms of classification accuracy. Results suggest that out of the three fusion approaches, only infrared image fusion (IVF) approach performed well with the considered dataset in terms of fusion indicators and classification accuracy. Keeping in view of its better performance, this study proposes a modified IVF approach by using different image pyramid methods. Comparison of results suggests an improved performance by modified IVF approach for the fusion of Sentinel 2 and Sentinel 1 data in comparison with the original IVF approach.