2023 · 43 citations · 63 references
ComputingComputer ScienceComputer VisionReal-time RenderingRenderingImage SynthesisScene ReconstructionHierarchical ClusteringInverse RenderingNeural Radiance FieldsNeural RenderingNovel View GenerationNovel View RenderingReflectanceView Synthesis
Existing inverse rendering combined with neural rendering methods can only perform editable novel view synthesis on object-specific scenes, while we present intrinsic neural radiance fields, dubbed IntrinsicNeRF, which introduce intrinsic decomposition into the NeRF-based neural rendering method and can extend its application to room-scale scenes. Since intrinsic decomposition is a fundamentally under-constrained inverse problem, we propose a novel distance-aware point sampling and adaptive reflectance iterative clustering optimization method, which enables IntrinsicNeRF with traditional intrinsic decomposition constraints to be trained in an unsupervised manner, resulting in multi-view consistent intrinsic decomposition results. To cope with the problem that different adjacent instances of similar reflectance in a scene are incorrectly clustered together, we further propose a hierarchical clustering method with coarse-to-fine optimization to obtain a fast hierarchical indexing representation. It supports compelling real-time augmented applications such as recoloring and illumination variation. Extensive experiments and editing samples on both object-specific/room-scale scenes and synthetic/real-word data demonstrate that we can obtain consistent intrinsic decomposition results and high-fidelity novel view synthesis even for challenging sequences.
63
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Richard Zhang, Phillip Isola, Alexei A. Efros et al. · 2018 · 12.4K citations
Proceedings European Conference on Computer Vision 2002
Anders Heyden, Gunnar Sparr, Mads Nielsen et al. · Lund University Publications (Lund University) · 2002 · 5.2K citations
A morphable model for the synthesis of 3D faces
Volker Blanz, Thomas Vetter · 1999 · 4.8K citations · Full text
Edwin H. Land, John J. McCann · Journal of the Optical Society of America · 1971 · 4K citations
Medicine, Physiology, Vision Science +14