2023 · 31 citations · 34 references
ComputingComputer ScienceComputer VisionMachine VisionRobot Vision3D RenderingImage SynthesisMotion EstimationOptical FlowConstraintsInverse Function TheoremMulti-view SynthesisNovel View GenerationObject Motion AnalysisObject Motion ModelingScene FlowView Synthesis
In this paper we present a new method for deformable NeRF that can directly use optical flow as supervision. We overcome the major challenge with respect to the computationally inefficiency of enforcing the flow constraints to the backward deformation field, used by deformable NeRFs. Specifically, we show that inverting the backward deformation function is actually not needed for computing scene flows between frames. This insight dramatically simplifies the problem, as one is no longer constrained to deformation functions that can be analytically inverted. Instead, thanks to the weak assumptions required by our derivation based on the inverse function theorem, our approach can be extended to a broad class of commonly used backward deformation field. We present results on monocular novel view synthesis with rapid object motion, and demonstrate significant improvements over baselines without flow supervision.
34
Image quality assessment: from error visibility to structural similarity
Zhou Wang, Alan C. Bovik, Hamid R. Sheikh et al. · IEEE Transactions on Image Processing · 2004 · 56K citations
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Richard Zhang, Phillip Isola, Alexei A. Efros et al. · 2018 · 12.4K citations
Structure-from-Motion Revisited
Johannes L. Schönberger, Jan‐Michael Frahm · 2016 · 5.8K citations
Matthew Loper, Naureen Mahmood, Javier Romero et al. · ACM Transactions on Graphics · 2015 · 3.6K citations
Architecture, Anatomy, 3D Modeling +15