Proceedings of the AAAI Conference on Artificial Intelligence · 2023 · 13 citations · 42 references
ComputingVision Science3D ModelingComputer VisionRendering3D ReconstructionGeometry ProcessingMotion CaptureMulti-view GeometryScene ReconstructionScene UnderstandingHuman Body ReconstructionHuman Pose EstimationImage CalibrationMulti-view 3D ReconstructionMulti-view ReconstructionScene Reconstruction From Multiple ViewsViewpoint Estimation
To date, little attention has been given to multi-view 3D human mesh estimation, despite real-life applicability (e.g., motion capture, sport analysis) and robustness to single-view ambiguities. Existing solutions typically suffer from poor generalization performance to new settings, largely due to the limited diversity of image/3D-mesh pairs in multi-view training data. To address this shortcoming, people have explored the use of synthetic images. But besides the usual impact of visual gap between rendered and target data, synthetic-data-driven multi-view estimators also suffer from overfitting to the camera viewpoint distribution sampled during training which usually differs from real-world distributions. Tackling both challenges, we propose a novel simulation-based training pipeline for multi-view human mesh recovery, which (a) relies on intermediate 2D representations which are more robust to synthetic-to-real domain gap; (b) leverages learnable calibration and triangulation to adapt to more diversified camera setups; and (c) progressively aggregates multi-view information in a canonical 3D space to remove ambiguities in 2D representations. Through extensive benchmarking, we demonstrate the superiority of the proposed solution especially for unseen in-the-wild scenarios.
42
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
Shih-En Wei, Varun Ramakrishna, Takeo Kanade et al. · 2016 · 2.8K citations
End-to-End Recovery of Human Shape and Pose
Angjoo Kanazawa, Michael J. Black, David W. Jacobs et al. · 2018 · 1.9K citations
Monocular 3D Human Pose Estimation in the Wild Using Improved CNN Supervision
Dushyant Mehta, Helge Rhodin, Dan Casas et al. · 2017 · 1.2K citations