A solvable model of learning generative diffusion: theory and insights

Hugo Cui, Cengiz Pehlevan, Yue M. Lu
1/7/2025
cs.LGcond-mat.dis-nn

No official code found. Best implementation: HugoCui/AE_diffusion (1 star, last commit 2025-01-20).

Abstract

In this manuscript, we consider the problem of learning a flow or diffusion-based generative model parametrized by a two-layer auto-encoder, trained with online stochastic gradient descent, on a high-dimensional target density with an underlying low-dimensional manifold structure. We derive a tight asymptotic characterization of low-dimensional projections of the distribution of samples generated by the learned model, ascertaining in particular its dependence on the number of training samples. Building on this analysis, we discuss how mode collapse can arise, and lead to model collapse when the generative model is re-trained on generated synthetic data.

View on arXivView PDF

Code Implementations(1)

HugoCui/AE_diffusionCommunity100%

Code for the paper : architectural bias in transport-based generative models - precise asymptotics

11Jupyter NotebookJan 3, 20251 years ago

Similar Papers

Analysis of learning a flow-based generative model from limited sample complexity

86% match1 repo22026

Diffusion-Model-Assisted Supervised Learning of Generative Models for Density Estimation

86% match2023

Heat Death of Generative Models in Closed-Loop Learning

86% match2024

Latent Schr{ö}dinger Bridge Diffusion Model for Generative Learning

86% match2024

Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connections

85% match1 repo42024

Generative Learning of Densities on Manifolds

85% match2025

Cite this paper

@article{cui2025solvable,
  title  = {A solvable model of learning generative diffusion: theory and insights},
  author = {Hugo Cui and Cengiz Pehlevan and Yue M. Lu},
  year   = {2025},
  eprint = {2501.03937},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2501.03937v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2501.03937.svg)](https://codeofpaper.com/paper/2501.03937)

Discussion