Proceedings of the 30th ACM International Conference on Multimedia · 2022 · 18 citations · 39 references
ArtsVision ScienceComputer VisionPhotographyRendering3D RenderingComputational PhotographyImage EditingPhotorealistic RenderingRendering PipelineScene ReconstructionNeural RenderingNovel View GenerationNovel View RenderingOutdoor Scene ReconstructionPhoto-realistic RenderingRealism EnhancementScene Reconstruction From ImagesScene Representation
Expanding an existing tourist photo from a partially captured scene to a full scene is one of the desired experiences for photography applications. Although photo extrapolation has been well studied, it is much more challenging to extrapolate a photo (i.e., selfie) from a narrow field of view to a wider one while maintaining a similar visual style. In this paper, we propose a factorized neural re-rendering model to produce photorealistic novel views from cluttered outdoor Internet photo collections, which enables the applications including controllable scene re-rendering, photo extrapolation and even extrapolated 3D photo generation. Specifically, we first develop a novel factorized re-rendering pipeline to handle the ambiguity in the decomposition of geometry, appearance and illumination. We also propose a composited training strategy to tackle the unexpected occlusion in Internet images. Moreover, to enhance photo-realism when extrapolating tourist photographs, we propose a novel realism augmentation process to complement appearance details, which automatically propagates the texture details from a narrow captured photo to the extrapolated neural rendered image. The experiments and photo editing examples on outdoor scenes demonstrate the superior performance of our proposed method in both photo-realism and downstream applications. Code and the supplementary material are available on the project webpage: https://zju3dv.github.io/neural_outdoor_rerender/.
39
Advances in neural information processing systems 7
Computers & Mathematics with Applications · 1996 · 14.4K citations
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Richard Zhang, Phillip Isola, Alexei A. Efros et al. · 2018 · 12.4K citations
Structure-from-Motion Revisited
Johannes L. Schönberger, Jan‐Michael Frahm · 2016 · 5.8K citations