GHARP: Real-time Gaussian Head Animation from Large-scale Reconstruction Prior

Key points

• We decouple the problem into an identity stage that builds a representation of the subject's geometry and appearance offline, and an animation stage that predicts expression-dependent residuals on top of it at runtime.1
• Our method performs animation in a semantically structured latent space of a pretrained reconstruction model, where expression changes remain spatially contained, making residual prediction efficient.1
• This reconstruction prior provides a consistent spatial layout, allowing fusion of multiple input views into a compact, fixed-size canonical Gaussian representation.1
• We address this with a body alignment network that learns to align the person's body in the target image with the input reference images, removing the ambiguity from the training signal.1

Sources (1)

• [1]GHARP: Real-time Gaussian Head Animation from Large-scale Reconstruction Prior
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 09:50 PM

  We present GHARP (Real-time Gaussian Head Animation from Large-scale Reconstruction Prior), a method that animates 3D human heads in real time from a few input images of a subject and a driving expression signal.

  We decouple the problem into an identity stage that builds a representation of the subject's geometry and appearance offline, and an animation stage that predicts expression-dependent residuals on top of it at runtime.

Extractive summary: sentences quoted from the sources.