An advanced image fusion algorithm based on wavelet transform: incorporation with PCA and morphological processing

Abstract

There are numerous applications for image fusion, some of which include medical imaging, remote sensing, nighttime operations and multi-spectral imaging. In general, the discrete wavelet transform (DWT) and various pyramids (such as Laplacian, ratio, contrast, gradient and morphological pyramids) are the most common and effective methods. For quantitative evaluation of the quality of fused imagery, the root mean square error (RMSE) is the most suitable measure of quality if there is a “ground truth” image available; otherwise, the entropy, spatial frequency or image quality index of the input images and the fused images can be calculated and compared. Here, after analyzing the pyramids’ performance with the four measures mentioned, an advanced wavelet transform (aDWT) method that incorporates principal component analysis (PCA) and morphological processing into a regular DWT fusion algorithm is presented. Specifically, at each scale of the wavelet transformed images, a principle vector was derived from two input images and then applied to two of the images’ approximation coefficients (i.e., they were fused by using the principal eigenvector). For the detail coefficients (i.e., three quarters of the coefficients), the larger absolute values were chosen and subjected to a neighborhood morphological processing procedure which served to verify the selected pixels by using a “filling” and “cleaning” operation (this operation filled or removed isolated pixels in a 3-by-3 local region). The fusion performance of the advanced DWT (aDWT) method proposed here was compared with six other common methods, and, based on the four quantitative measures, was found to perform the best when tested on the four input image types. Since the different image sources used here varied with respect to intensity, contrast, noise, and intrinsic characteristics, the aDWT is a promising image fusion procedure for inhomogeneous imagery.