Light Field Reconstruction Using Efficient Pseudo 4D Epipolar-Aware Structure

Abstract

Limited by sensor resolution, Light Field (LF) images often suffer from an inherent trade-off between angular resolution and spatial resolution. LF reconstruction, including Spatial Super-Resolution (LFSSR) and Angular Super-Resolution (LFASR), upsamples LF in spatial and angular domain based on the complementary information in different views. In this paper, we propose efficient pseudo-4D end-to-end frameworks for LFSSR and LFASR, respectively. Since the epipolar information and spatial-angular information reflect the relationship between views, we propose to fully consider both of them to exploit complementary information thoroughly. Specifically, we first extract epipolar information from horizontally and vertically stacked views separately. Then an Epipolar-Aware Grid (EAG) network composed of Dual Interactive Blocks (DIBs) fully interacts the epipolar information, characterizing the grid-like LF parallax structure. In order to effectively achieve dense interaction between the spatial and angular domain to extract complementary information, several Parallel Spatial-Angular Integration Blocks (PSAIBs) are further introduced. For LFs with large baselines, we also propose a generic shear attention network, which generalizes the model designed for small baselines to large baselines without depth estimation. As compared to the state-of-the-art methods, extensive experiments conducted on both synthetic and real-world datasets demonstrate our superiority in LFSSR tasks and LFASR tasks.