Multi-frame super-resolution algorithm for complex motion patterns

Abstract

Multi-frame super-resolution algorithms offer resolution enhancement for sequences of images with sampling limited resolution. However, classical approaches have been constrained by the accuracy of motion estimation while nonlocal approaches that use implicit motion estimation have attained only modest resolution improvement. In this paper, we propose a new multi-frame optical flow based super-resolution algorithm, which provides significant resolution enhancement for image sequences containing complex motion. The algorithm uses the standard camera image formation model and a variational super-resolution formulation with an anisotropic smoothness term adapting to local image structures. The key elements enabling super-resolution of complex motion patterns are the computation of two-way optical flow between the images and use of two corresponding uncertainty measures that approximate the optical flow interpolation error. Using the developed algorithm, we are able to demonstrate super-resolution of images for which optical flow estimation experiences near breakdown, due to the complexity of the motion patterns and the large magnitudes of the displacements. In comparison, we show that for these images some conventional super-resolution approaches fail, while others including nonlocal super-resolution technique produce distortions and provide lower (1-1.8 dB) image quality enhancement compared to the proposed algorithm.