2023 · 17 citations · 36 references
ArchitectureComputer ScienceComputer VisionDigital RenderingRendering3D Animation3D RenderingAnimationMotion CaptureRendering PipelineVolumetric RenderingDynamics Model LearningHair AnimationHair Appearance ModelingLatent Space ModelsNovel View GenerationNovel View RenderingView SynthesisVirtual Avatars
The capture and animation of human hair are two of the major challenges in the creation of realistic avatars for the virtual reality. Both problems are highly challenging, because hair has complex geometry and appearance and exhibits challenging motion. In this paper, we present a two-stage approach that models hair independently of the head to address these challenges in a data-driven manner. The first stage, state compression, learns a low-dimensional latent space of 3D hair states including motion and appearance via a novel autoencoder-as-a-tracker strategy. To better disentangle the hair and head in appearance learning, we employ multi-view hair segmentation masks in combination with a differentiable volumetric renderer. The second stage optimizes a novel hair dynamics model that performs temporal hair transfer based on the discovered latent codes. To enforce higher stability while driving our dynamics model, we employ the 3D point-cloud autoencoder from the compression stage for denoising of the hair state. Our model outperforms the state of the art in novel view synthesis and is capable of creating novel hair animations without relying on hair observations as a driving signal. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">†</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">†</sup> Project page at https://ziyanwl.github.io/neuwigs/.
36
Auto-Encoding Variational Bayes
Diederik P. Kingma, Max Welling · UvA-DARE (University of Amsterdam) · 2013 · 15.6K citations · Full text
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Richard Zhang, Phillip Isola, Alexei A. Efros et al. · 2018 · 12.4K citations
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Raffaelli Charles, Hao Su, Kaichun Mo et al. · 2017 · 9.8K citations
A morphable model for the synthesis of 3D faces
Volker Blanz, Thomas Vetter · 1999 · 4.8K citations · Full text
Learning a model of facial shape and expression from 4D scans
Tianye Li, Timo Bolkart, Michael J. Black et al. · ACM Transactions on Graphics · 2017 · 1.1K citations