Multi-Exposure Image Fusion by Optimizing A Structural Similarity Index

Abstract

We propose a multi-exposure image fusion (MEF) algorithm by optimizing a novel objective quality measure, namely the color MEF structural similarity (MEF-SSIM <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> ) index. The design philosophy we introduce here is substantially different from existing ones. Instead of pre-defining a systematic computational structure for MEF (e.g., multiresolution transformation and transform domain fusion followed by image reconstruction), we directly operate in the space of all images, searching for the image that optimizes MEF-SSIM <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> . Specifically, we first construct the MEF-SSIM <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> index by improving upon and expanding the application scope of the existing MEF-SSIM algorithm. We then describe a gradient ascent-based algorithm, which starts from any initial point in the space of all possible images and iteratively moves towards the direction that improves MEF-SSIM <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> until convergence. Numerical and subjective experiments demonstrate that the proposed algorithm consistently produces better quality fused images both visually and in terms of MEF-SSIM <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> . The final high quality fused image appears to have little dependence on the initial image. The proposed optimization framework is readily extensible to construct better MEF algorithms when better objective quality models for MEF are available.