Multi-sensor image data fusion based on pixel-level weights of wavelet and the PCA transform

Abstract

The goal of image fusion is to create new images that are more suitable for the purposes of human visual perception, object detection and target recognition. For automatic target recognition (ATR), we can use multi-sensor data including visible and infrared images to increase the recognition rate. In this paper, we propose a new multiresolution data fusion scheme based on the principal component analysis (PCA) transform and the pixel-level weights wavelet transform including thermal weights and visual weights. In order to get a more ideal fusion result, a linear local mapping which based on the PCA is used to create a new "origin" image of the image fusion. We use multiresolution decompositions to represent the input images at different scales, present a multiresolution/multimodal segmentation to partition the image domain at these scales. The crucial idea is to use this segmentation to guide the fusion process. Physical thermal weights and perceptive visual weights are used as segmentation multimodals. Daubechies wavelet is choosen as the wavelet basis. Experimental results confirm that the proposed algorithm is the best image sharpening method and can best maintain the spectral information of the original infrared image. Also, the proposed technique performs better than the other ones in the literature, more robust and effective, from both subjective visual effects and objective statistical analysis results.