PersonaHOI: Effortlessly Improving Personalized Face with Human-Object Interaction Generation

Xinting Hu, Haoran Wang, Jan Eric Lenssen, Bernt Schiele
1/10/2025
cs.CV

Official code: JoyHuYY1412/PersonaHOI (5 stars, last commit 2025-01-13).

Abstract

We introduce PersonaHOI, a training- and tuning-free framework that fuses a general StableDiffusion model with a personalized face diffusion (PFD) model to generate identity-consistent human-object interaction (HOI) images. While existing PFD models have advanced significantly, they often overemphasize facial features at the expense of full-body coherence, PersonaHOI introduces an additional StableDiffusion (SD) branch guided by HOI-oriented text inputs. By incorporating cross-attention constraints in the PFD branch and spatial merging at both latent and residual levels, PersonaHOI preserves personalized facial details while ensuring interactive non-facial regions. Experiments, validated by a novel interaction alignment metric, demonstrate the superior realism and scalability of PersonaHOI, establishing a new standard for practical personalized face with HOI generation. Our code will be available at https://github.com/JoyHuYY1412/PersonaHOI

View on arXivView PDF

Code Implementations(1)

JoyHuYY1412/PersonaHOIOfficial100%
50Jan 9, 20251 years ago

Similar Papers

CapHuman: Capture Your Moments in Parallel Universes

87% match1 repo992026

DynamicID: Zero-Shot Multi-ID Image Personalization with Flexible Facial Editability

87% match1 repo362025

Training for Identity, Inference for Controllability: A Unified Approach to Tuning-Free Face Personalization

87% match1 repo102025

Dense-Face: Personalized Face Generation Model via Dense Annotation Prediction

86% match2024

InstaFace: Identity-Preserving Facial Editing with Single Image Inference

85% match2025

From Parts to Whole: A Unified Reference Framework for Controllable Human Image Generation

85% match1 repo1962024

Cite this paper

@article{hu2025personahoi,
  title  = {PersonaHOI: Effortlessly Improving Personalized Face with Human-Object Interaction Generation},
  author = {Xinting Hu and Haoran Wang and Jan Eric Lenssen and Bernt Schiele},
  year   = {2025},
  eprint = {2501.05823},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2501.05823v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2501.05823.svg)](https://codeofpaper.com/paper/2501.05823)

Discussion