Universal Guidance for Diffusion Models

Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum +2 more
2/14/2023
cs.CVcs.LG

Official code: arpitbansal297/Universal-Guided-Diffusion (509 stars, last commit 2023-07-11). 1 more implementation below.

Abstract

Typical diffusion models are trained to accept a particular form of conditioning, most commonly text, and cannot be conditioned on other modalities without retraining. In this work, we propose a universal guidance algorithm that enables diffusion models to be controlled by arbitrary guidance modalities without the need to retrain any use-specific components. We show that our algorithm successfully generates quality images with guidance functions including segmentation, face recognition, object detection, and classifier signals. Code is available at https://github.com/arpitbansal297/Universal-Guided-Diffusion.

View on arXivView PDF

Code Implementations(2)

arpitbansal297/Universal-Guided-DiffusionOfficial100%
50940Shell, PythonJan 30, 20233 years ago
VladPyzh/Universal-Guided-DiffusionCommunity63%

Reimplementation of https://arxiv.org/abs/2302.07121

10Python, Jupyter NotebookFeb 19, 20251 years ago

Similar Papers

Diffusion Models without Classifier-free Guidance

87% match1 repo1752025

GeoGuide: Geometric guidance of diffusion models

87% match1 repo22024

Understanding and Improving Training-free Loss-based Diffusion Guidance

87% match1 repo192026

Elucidating The Design Space of Classifier-Guided Diffusion Generation

86% match2 repos882026

Guiding a diffusion model using sliding windows

86% match1 repo32024

TFG: Unified Training-Free Guidance for Diffusion Models

85% match2 repos882024

Cite this paper

@article{bansal2023universal,
  title  = {Universal Guidance for Diffusion Models},
  author = {Arpit Bansal and Hong-Min Chu and Avi Schwarzschild and Soumyadip Sengupta and Micah Goldblum and Jonas Geiping and Tom Goldstein},
  year   = {2023},
  eprint = {2302.07121},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2302.07121v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2302.07121.svg)](https://codeofpaper.com/paper/2302.07121)

Discussion