Face-MakeUp: Multimodal Facial Prompts for Text-to-Image Generation

Dawei Dai, Mingming Jia, Yinxiu Zhou, Hang Xing, Chenghang Li
1/5/2025
cs.CVcs.AI

Official code: ddw2AIGROUP2CQUPT/Face-MakeUp (26 stars, last commit 2025-01-19).

Abstract

Facial images have extensive practical applications. Although the current large-scale text-image diffusion models exhibit strong generation capabilities, it is challenging to generate the desired facial images using only text prompt. Image prompts are a logical choice. However, current methods of this type generally focus on general domain. In this paper, we aim to optimize image makeup techniques to generate the desired facial images. Specifically, (1) we built a dataset of 4 million high-quality face image-text pairs (FaceCaptionHQ-4M) based on LAION-Face to train our Face-MakeUp model; (2) to maintain consistency with the reference facial image, we extract/learn multi-scale content features and pose features for the facial image, integrating these into the diffusion model to enhance the preservation of facial identity features for diffusion models. Validation on two face-related test datasets demonstrates that our Face-MakeUp can achieve the best comprehensive performance.All codes are available at:https://github.com/ddw2AIGROUP2CQUPT/Face-MakeUp

View on arXivView PDF

Code Implementations(1)

ddw2AIGROUP2CQUPT/Face-MakeUpOfficial100%

Face-MakeUp (SD1.5): Multimodal Facial Prompts for Text-to-Image Generation （ECAI-2025）

261Shell, PythonDec 27, 20241 years ago
text-to-image-diffusioin-modeltext-to-image-generationtext-to-image-synthesis

Similar Papers

Dense-Face: Personalized Face Generation Model via Dense Annotation Prediction

88% match2024

UVMap-ID: A Controllable and Personalized UV Map Generative Model

88% match1 repo242024

Fusion Embedding for Pose-Guided Person Image Synthesis with Diffusion Model

88% match1 repo2024

ExpertGen: Training-Free Expert Guidance for Controllable Text-to-Face Generation

87% match2025

ID3: Identity-Preserving-yet-Diversified Diffusion Models for Synthetic Face Recognition

87% match2026

DiffusionRig: Learning Personalized Priors for Facial Appearance Editing

86% match1 repo2952026

Cite this paper

@article{dai2025facemakeup,
  title  = {Face-MakeUp: Multimodal Facial Prompts for Text-to-Image Generation},
  author = {Dawei Dai and Mingming Jia and Yinxiu Zhou and Hang Xing and Chenghang Li},
  year   = {2025},
  eprint = {2501.02523},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2501.02523v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2501.02523.svg)](https://codeofpaper.com/paper/2501.02523)

Discussion