ACM Transactions on Graphics · 2024 · 14 citations · 18 references
ComputingComputer Science3D ModelingComputer VisionMachine VisionBody RepresentationFace RecognitionMotion CaptureScene ReconstructionHand Motion CaptureHuman Body ReconstructionMotion Capture Performance EvaluationMotion Capture ProcessingShape
We tackle the problem of highly-accurate, holistic performance capture for the face, body and hands simultaneously. Motion-capture technologies used in film and game production typically focus only on face, body or hand capture independently, involve complex and expensive hardware and a high degree of manual intervention from skilled operators. While machine-learning-based approaches exist to overcome these problems, they usually only support a single camera, often operate on a single part of the body, do not produce precise world-space results, and rarely generalize outside specific contexts. In this work, we introduce the first technique for markerfree, high-quality reconstruction of the complete human body, including eyes and tongue, without requiring any calibration, manual intervention or custom hardware. Our approach produces stable world-space results from arbitrary camera rigs as well as supporting varied capture environments and clothing. We achieve this through a hybrid approach that leverages machine learning models trained exclusively on synthetic data and powerful parametric models of human shape and motion. We evaluate our method on a number of body, face and hand reconstruction benchmarks and demonstrate state-of-the-art results that generalize on diverse datasets.
18
Martin A. Fischler, Robert C. Bolles · Communications of the ACM · 1981 · 25.5K citations · Full text
ImageNet: A large-scale hierarchical image database
Jia Deng, Wei Dong, R. Socher et al. · 2009 IEEE Conference on Computer Vision and Pattern Recognition · 2009 · 2.9K citations
End-to-End Recovery of Human Shape and Pose
Angjoo Kanazawa, Michael J. Black, David W. Jacobs et al. · 2018 · 1.9K citations
Dragomir Anguelov, Daphne Koller, Sebastian Thrun et al. · ACM Transactions on Graphics · 2005 · 1.6K citations