2020 · 64 citations · 26 references
ComputingComputer ScienceComputer VisionMachine VisionRendering3D ReconstructionDepth EstimationOptical FlowScene ReconstructionScene UnderstandingStereo VisionConvolutional Neural NetworksDeep Optical FlowDepth Map ReconstructionInpaintingLearning-based Optical FlowLight Field ReconstructionOptical Flow EstimationViews
In this paper, we present a learning-based framework for light field view synthesis from a subset of input views. Building upon a light-weight optical flow estimation network to obtain depth maps, our method employs two reconstruction modules in pixel and feature domains respectively. For the pixel-wise reconstruction, occlusions are explicitly handled by a disparity-dependent interpolation filter, whereas inpainting on disoccluded areas is learned by convolutional layers. Due to disparity inconsistencies, the pixel-based reconstruction may lead to blurriness in highly textured areas as well as on object contours. On the contrary, the feature-based reconstruction well performs on high frequencies, making the reconstruction in the two domains complementary. End-to-end learning is finally performed including a fusion module merging pixel and feature-based reconstructions. Experimental results show that our method achieves state-of-the-art performance on both synthetic and real-world datasets, moreover, it is even able to extend light fields' baseline by extrapolating high quality views without additional training.
26
Marc Levoy, Pat Hanrahan · 1996 · 3.7K citations · Full text
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Eddy Ilg, N. Michael Mayer, Tonmoy Saikia et al. · 2017 · 3.3K citations
PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume
Deqing Sun, Xiaodong Yang, Ming-Yu Liu et al. · 2018 · 2.7K citations
Steven J. Gortler, Radek Grzeszczuk, Richard Szeliski et al. · 1996 · 2.5K citations · Full text