Portrait4D: Learning One-Shot 4D Head Avatar Synthesis using Synthetic Data

Abstract

Existing one-shot 4D head synthesis methods usually learn from monocular videos with the aid of 3DMM re-construction, yet the latter is evenly challenging which re-stricts them from reasonable 4D head synthesis. We present a method to learn one-shot 4D head synthesis via large-scale synthetic data. The key is to first learn a part-wise 4D generative model from monocular images via adver-sarial learning, to synthesize multi-view images of diverse identities and full motions as training data; then leverage a transformer-based animatable triplane reconstructor to learn 4D head reconstruction using the synthetic data. A novel learning strategy is enforced to enhance the general-izability to real images by disentangling the learning pro-cess of 3D reconstruction and reenactment. Experiments demonstrate our superiority over the prior art.