2021 · 219 citations · 47 references
ComputingRobotics Engineering3D ModelingComputer VisionRobot LearningBody RepresentationImage RegistrationPose EstimationShape DeformationSurface ReconstructionBody Pose EstimationHuman Body ReconstructionHuman Pose EstimationImplicit Function RepresentationMesh RegistrationNon-rigid RegistrationSurface RegistrationTransformations
We present SCANimate, an end-to-end trainable framework that takes raw 3D scans of a clothed human and turns them into an animatable avatar. These avatars are driven by pose parameters and have realistic clothing that moves and deforms naturally. SCANimate does not rely on a customized mesh template or surface mesh registration. We observe that fitting a parametric 3D body model, like SMPL, to a clothed human scan is tractable while surface registration of the body topology to the scan is often not, because clothing can deviate significantly from the body shape. We also observe that articulated transformations are invertible, resulting in geometric cycle-consistency in the posed and unposed shapes. These observations lead us to a weakly supervised learning method that aligns scans into a canonical pose by disentangling articulated deformations without template-based surface registration. Furthermore, to complete missing regions in the aligned scans while modeling pose-dependent deformations, we introduce a locally pose-aware implicit function that learns to complete and model geometry with learned pose correctives. In contrast to commonly used global pose embeddings, our local pose conditioning significantly reduces long-range spurious correlations and improves generalization to unseen poses, especially when training data is limited. Our method can be applied to pose-aware appearance modeling to generate a fully textured avatar. We demonstrate our approach on various clothing types with different amounts of training data, outperforming existing solutions and other variants in terms of fidelity and generality in every setting. The code is available at https://scanimate.is.tue.mpg.de.
47
Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks
Jun-Yan Zhu, Taesung Park, Phillip Isola et al. · 2017 · 21.9K citations · Full text
Marching cubes: A high resolution 3D surface construction algorithm
William E. Lorensen, H. E. Cline · 1987 · 10.2K citations · Full text
Computing, Radiology, Visualization +17
Matthew Loper, Naureen Mahmood, Javier Romero et al. · ACM Transactions on Graphics · 2015 · 3.6K citations
Architecture, Anatomy, 3D Modeling +15
End-to-End Recovery of Human Shape and Pose
Angjoo Kanazawa, Michael J. Black, David W. Jacobs et al. · 2018 · 1.9K citations
Dragomir Anguelov, Daphne Koller, Sebastian Thrun et al. · ACM Transactions on Graphics · 2005 · 1.6K citations