Bellman Diffusion Models

Liam Schramm, Abdeslam Boularias
7/16/2024
cs.LGcs.RO

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

Diffusion models have seen tremendous success as generative architectures. Recently, they have been shown to be effective at modelling policies for offline reinforcement learning and imitation learning. We explore using diffusion as a model class for the successor state measure (SSM) of a policy. We find that enforcing the Bellman flow constraints leads to a simple Bellman update on the diffusion step distribution.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Bellman Diffusion: Generative Modeling as Learning a Linear Operator in the Distribution Space

84% match2024

Diffusion Models for Reinforcement Learning: A Survey

84% match2 repos6702023

Diffusion World Model: Future Modeling Beyond Step-by-Step Rollout for Offline Reinforcement Learning

84% match2024

Policy Representation via Diffusion Probability Model for Reinforcement Learning

83% match1 repo1322023

Learning a Diffusion Model Policy from Rewards via Q-Score Matching

83% match1 repo342026

On Bellman equations for continuous-time policy evaluation I: discretization and approximation

83% match2024

Cite this paper

@article{schramm2024bellman,
  title  = {Bellman Diffusion Models},
  author = {Liam Schramm and Abdeslam Boularias},
  year   = {2024},
  eprint = {2407.12163},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.12163v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.12163.svg)](https://codeofpaper.com/paper/2407.12163)

Discussion