CDLM: Consistency Diffusion Language Models For Faster Sampling

Minseo Kim, Chenfeng Xu, Coleman Hooper, Harman Singh, Ben Athiwaratkun +3 more
11/24/2025
cs.LGcs.CL

Official code: SqueezeAILab/CDLM (41 stars, MIT, last commit 2025-11-25).

Abstract

Diffusion Language Models (DLMs) offer a promising parallel generation paradigm but suffer from slow inference due to numerous refinement steps and the inability to use standard KV caching. We introduce CDLM (Consistency Diffusion Language Models), a training-based acceleration method that simultaneously tackles both bottlenecks. CDLM integrates consistency modeling to drastically reduce the number of required sampling steps by enabling multi-token finalization. Furthermore, we enforce a block-wise causal attention mask during fine-tuning, making the model fully compatible with KV caching. Experiments show CDLM achieves 3.6x-14.5x lower latency while maintaining competitive accuracy on math and coding tasks. The full training and evaluation code is available at https://github.com/SqueezeAILab/CDLM.

View on arXivView PDF

Code Implementations(1)

SqueezeAILab/CDLMOfficial100%
MIT

CDLM: Consistency Diffusion Language Models for Faster Sampling

410Shell, PythonNov 24, 202511 months agoMIT

Similar Papers

CD4LM: Consistency Distillation and aDaptive Decoding for Diffusion Language Models

91% match1 repo112026

Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding

88% match1 repo1.1k2025

dLLM: Simple Diffusion Language Modeling

86% match1 repo2.7k2026

Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principles

86% match1 repo432025

FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion

86% match2025

dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching

86% match1 repo2112025

Cite this paper

@article{kim2025cdlm,
  title  = {CDLM: Consistency Diffusion Language Models For Faster Sampling},
  author = {Minseo Kim and Chenfeng Xu and Coleman Hooper and Harman Singh and Ben Athiwaratkun and Ce Zhang and Kurt Keutzer and Amir Gholami},
  year   = {2025},
  eprint = {2511.19269},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2511.19269v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2511.19269.svg)](https://codeofpaper.com/paper/2511.19269)

Discussion