2023 · 21 citations · 42 references
ComputingComputer ScienceComputer VisionDigital RenderingRendering3D RenderingImage EditingImage SynthesisScene ChangesScene UnderstandingDynamic Scene RepresentationNeural Radiance FieldsNeural View SynthesisNovel View RenderingRadianceScene DynamicsScene RepresentationView Synthesis
Neural radiance fields (NeRF) achieve highly photorealistic novel-view synthesis, but it's a challenging problem to edit the scenes modeled by NeRF-based methods, especially for dynamic scenes. We propose editable neural radiance fields that enable end-users to easily edit dynamic scenes and even support topological changes. Input with an image sequence from a single camera, our network is trained fully automatically and models topologically varying dynamics using our picked-out surface key points. Then end-users can edit the scene by easily dragging the key points to desired new positions. To achieve this, we propose a scene analysis method to detect and initialize key points by considering the dynamics in the scene, and a weighted key points strategy to model topologically varying dynamics by joint key points and weights optimization. Our method supports intuitive multi-dimensional (up to 3D) editing and can generate novel scenes that are unseen in the input sequence. Experiments demonstrate that our method achieves high-quality editing on various dynamic scenes and outperforms the state-of-the-art. Our code and captured data are available at https://chengwei-zheng.github.io/EditableNeRF/
42
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Richard Zhang, Phillip Isola, Alexei A. Efros et al. · 2018 · 12.4K citations
Multiscale structural similarity for image quality assessment
Zhou Wang, Eero P. Simoncelli, Alan C. Bovik · 2004 · 5.9K citations
Structure-from-Motion Revisited
Johannes L. Schönberger, Jan‐Michael Frahm · 2016 · 5.8K citations
Matthew Loper, Naureen Mahmood, Javier Romero et al. · ACM Transactions on Graphics · 2015 · 3.6K citations
Architecture, Anatomy, 3D Modeling +15
Justus Thies, Michael Zollhöfer, Matthias Nießner · ACM Transactions on Graphics · 2019 · 1.4K citations