2021 IEEE/CVF International Conference on Computer Vision (ICCV) · 2021 · 185 citations · 44 references
ComputingDigital Design3D ModelingComputer VisionRendering3D Animation3D ReconstructionBody RepresentationMulti-view GeometryScene ReconstructionSurface ReconstructionHand-crafted FeaturesHuman Body ModelsHuman Body ReconstructionImage-to-image TranslationReconstruction Quality Assessment
We present ARCH++, an image-based method to reconstruct 3D avatars with arbitrary clothing styles. Our reconstructed avatars are animation-ready and highly realistic, in both the visible regions from input views and the unseen regions. While prior work shows great promise of reconstructing animatable clothed humans with various topologies, we observe that there exist fundamental limitations resulting in sub-optimal reconstruction quality. In this paper, we revisit the major steps of image-based avatar reconstruction and address the limitations with ARCH++. First, we introduce an end-to-end point based geometry encoder to better describe the semantics of the underlying 3D human body, in replacement of previous hand-crafted features. Second, in order to address the occupancy ambiguity caused by topological changes of clothed humans in the canonical pose, we propose a co-supervising framework with cross-space consistency to jointly estimate the occupancy in both the posed and canonical spaces. Last, we use image-to-image translation networks to further refine detailed geometry and texture on the reconstructed surface, which improves the fidelity and consistency across arbitrary viewpoints. In the experiments, we demonstrate improvements over the state of the art on both public benchmarks and user studies in reconstruction quality and realism.
44
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Raffaelli Charles, Hao Su, Kaichun Mo et al. · 2017 · 9.8K citations
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu et al. · 2018 · 4.4K citations
Matthew Loper, Naureen Mahmood, Javier Romero et al. · ACM Transactions on Graphics · 2015 · 3.6K citations
Architecture, Anatomy, 3D Modeling +15
Marcelo Bertalmı́o, Guillermo Sapiro, V. Caselles et al. · 2000 · 3.6K citations · Full text