2013 · 215 citations · 29 references
ComputingComputer Science3D ModelingComputational GeometryComputer Vision3D ReconstructionScene ReconstructionShape DeformationShape RepresentationSurface ReconstructionVariational FormulationDense 3D ReconstructionDense ReconstructionGlobal Energy MinimizationNon-rigid ReconstructionNon-rigid Surface ReconstructionRegularizationTotal Variation MinimizationVariational Reconstruction
This paper offers the first variational approach to the problem of dense 3D reconstruction of non-rigid surfaces from a monocular video sequence. We formulate non-rigid structure from motion (nrsfm) as a global variational energy minimization problem to estimate dense low-rank smooth 3D shapes for every frame along with the camera motion matrices, given dense 2D correspondences. Unlike traditional factorization based approaches to nrsfm, which model the low-rank non-rigid shape using a fixed number of basis shapes and corresponding coefficients, we minimize the rank of the matrix of time-varying shapes directly via trace norm minimization. In conjunction with this low-rank constraint, we use an edge preserving total-variation regularization term to obtain spatially smooth shapes for every frame. Thanks to proximal splitting techniques the optimization problem can be decomposed into many point-wise sub-problems and simple linear systems which can be easily solved on GPU hardware. We show results on real sequences of different objects (face, torso, beating heart) where, despite challenges in tracking, illumination changes and occlusions, our method reconstructs highly deforming smooth surfaces densely and accurately directly from video, without the need for any prior models or shape templates.
29
A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms
Steven M. Seitz, Brian Curless, James Diebel et al. · 2006 · 2.4K citations