Variational Light Field Analysis for Disparity Estimation and Super-Resolution

TLDR

The paper proposes a continuous variational framework for 4D light field analysis, enabling disparity reconstruction and spatial/angular super‑resolution. The method estimates disparity maps locally via epipolar plane image analysis, then uses them in a continuous inverse formulation to synthesize super‑resolved novel views, solved with convex relaxation and evaluated against multiview stereo on real and benchmark light‑field data. The approach achieves fast, subpixel‑accurate disparity estimation and outperforms multiview stereo in speed and accuracy, with datasets and code publicly available.

Abstract

We develop a continuous framework for the analysis of 4D light fields, and describe novel variational methods for disparity reconstruction as well as spatial and angular super-resolution. Disparity maps are estimated locally using epipolar plane image analysis without the need for expensive matching cost minimization. The method works fast and with inherent subpixel accuracy since no discretization of the disparity space is necessary. In a variational framework, we employ the disparity maps to generate super-resolved novel views of a scene, which corresponds to increasing the sampling rate of the 4D light field in spatial as well as angular direction. In contrast to previous work, we formulate the problem of view synthesis as a continuous inverse problem, which allows us to correctly take into account foreshortening effects caused by scene geometry transformations. All optimization problems are solved with state-of-the-art convex relaxation techniques. We test our algorithms on a number of real-world examples as well as our new benchmark data set for light fields, and compare results to a multiview stereo method. The proposed method is both faster as well as more accurate. Data sets and source code are provided online for additional evaluation.