The Journey, Not the Destination: How Data Guides Diffusion Models

Kristian Georgiev, Joshua Vendrow, Hadi Salman, Sung Min Park, Aleksander Madry
12/11/2023
cs.CVcs.LG

Official code: MadryLab/journey-TRAK (25 stars, MIT, last commit 2023-12-12).

Abstract

Diffusion models trained on large datasets can synthesize photo-realistic images of remarkable quality and diversity. However, attributing these images back to the training data-that is, identifying specific training examples which caused an image to be generated-remains a challenge. In this paper, we propose a framework that: (i) provides a formal notion of data attribution in the context of diffusion models, and (ii) allows us to counterfactually validate such attributions. Then, we provide a method for computing these attributions efficiently. Finally, we apply our method to find (and evaluate) such attributions for denoising diffusion probabilistic models trained on CIFAR-10 and latent diffusion models trained on MS COCO. We provide code at https://github.com/MadryLab/journey-TRAK .

View on arXivView PDF

Code Implementations(1)

MadryLab/journey-TRAKOfficial100%
MIT

Code for the paper "The Journey, Not the Destination: How Data Guides Diffusion Models"

252Python2 years agoMIT
data-attributiondata-valuationdiffusion-modelsgenerative-models

Similar Papers

Diffusion Attribution Score: Evaluating Training Data Influence in Diffusion Model

91% match2026

Diffusion Attribution Score: Evaluating Training Data Influence in Diffusion Models

91% match1 repo62024

Attributing HOW, Not Just WHICH: Counterfactual Response Trajectories for Diffusion Models

88% match2026

Intriguing Properties of Data Attribution on Diffusion Models

88% match1 repo382026

Tracing the Roots: Leveraging Temporal Dynamics in Diffusion Trajectories for Origin Attribution

87% match2024

Training Data Attribution for Diffusion Models

87% match2 repos62023

Cite this paper

@article{georgiev2023journey,
  title  = {The Journey, Not the Destination: How Data Guides Diffusion Models},
  author = {Kristian Georgiev and Joshua Vendrow and Hadi Salman and Sung Min Park and Aleksander Madry},
  year   = {2023},
  eprint = {2312.06205},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2312.06205v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2312.06205.svg)](https://codeofpaper.com/paper/2312.06205)

Discussion