Multimodal Conditional 3D Face Geometry Generation

Christopher Otto, Prashanth Chandran, Sebastian Weiss, Markus Gross, Gaspard Zoss +1 more
7/1/2024
cs.CVcs.GR

No confident code implementation found for this paper.

Abstract

We present a new method for multimodal conditional 3D face geometry generation that allows user-friendly control over the output identity and expression via a number of different conditioning signals. Within a single model, we demonstrate 3D faces generated from artistic sketches, portrait photos, Canny edges, FLAME face model parameters, 2D face landmarks, or text prompts. Our approach is based on a diffusion process that generates 3D geometry in a 2D parameterized UV domain. Geometry generation passes each conditioning signal through a set of cross-attention layers (IP-Adapter), one set for each user-defined conditioning signal. The result is an easy-to-use 3D face generation tool that produces topology-consistent, high-quality geometry with fine-grain user control.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Single Image, Any Face: Generalisable 3D Face Generation

84% match2024

Diffusion-Driven GAN Inversion for Multi-Modal Face Image Generation

84% match1 repo72026

IMAGPose: A Unified Conditional Framework for Pose-Guided Person Generation

83% match1 repo1882026

FPGA: Flexible Portrait Generation Approach

83% match2024

MM2Latent: Text-to-facial image generation and editing in GANs with multimodal assistance

82% match1 repo202024

Controllable Face Synthesis with Semantic Latent Diffusion Models

82% match1 repo152024

Cite this paper

@article{otto2024multimodal,
  title  = {Multimodal Conditional 3D Face Geometry Generation},
  author = {Christopher Otto and Prashanth Chandran and Sebastian Weiss and Markus Gross and Gaspard Zoss and Derek Bradley},
  year   = {2024},
  eprint = {2407.01074},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.01074v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.01074.svg)](https://codeofpaper.com/paper/2407.01074)

Discussion