2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · 2022 · 60 citations · 40 references
ComputingVisualization3D ModelingCinematographyComputer Vision3D AnimationAnimationCharacter AnimationMotion CapturePose EstimationShape DeformationCanonical RepresentationDeformationDeformation Field ReconstructionHuman Pose EstimationImplicit Surface RepresentationNon-rigid ReconstructionRgb-d ReconstructionVirtual Avatars
We present a novel method to learn Personalized Implicit Neural Avatars (PINA) from a short RGB-D sequence. This allows non-expert users to create a detailed and personal-ized virtual copy of themselves, which can be animated with realistic clothing deformations. PINA does not require complete scans, nor does it require a prior learned from large datasets of clothed humans. Learning a complete avatar in this setting is challenging, since only few depth observations are available, which are noisy and incomplete (i.e. only partial visibility of the body per frame). We propose a method to learn the shape and non-rigid deformations via a pose-conditioned implicit surface and a deformation field, defined in canonical space. This allows us to fuse all partial observations into a single consistent canonical representation. Fusion is formulated as a global optimization problem over the pose, shape and skinning parameters. The method can learn neural avatars from real noisy RGB-D sequences for a diverse set of people and clothing styles and these avatars can be animated given unseen motion sequences.
40
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik et al. · Communications of the ACM · 2021 · 5.8K citations · Full text
KinectFusion: Real-time dense surface mapping and tracking
Richard A. Newcombe, Andrew Fitzgibbon, Shahram Izadi et al. · 2011 · 3.9K citations
Matthew Loper, Naureen Mahmood, Javier Romero et al. · ACM Transactions on Graphics · 2015 · 3.6K citations
Architecture, Anatomy, 3D Modeling +15
Learning Implicit Fields for Generative Shape Modeling
Zhiqin Chen, Hao Zhang · 2019 · 1.5K citations
PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization
Shunsuke Saito, Zeng Huang, Ryota Natsume et al. · 2019 · 1.2K citations