CodeFusion: A Pre-trained Diffusion Model for Code Generation

Mukul Singh, José Cambronero, Sumit Gulwani, Vu Le, Carina Negreanu +1 more
10/26/2023
cs.SEcs.AIcs.CL

No confident code implementation found for this paper. 6 possible matches can be shown below.

Abstract

Imagine a developer who can only change their last line of code, how often would they have to start writing a function from scratch before it is correct? Auto-regressive models for code generation from natural language have a similar limitation: they do not easily allow reconsidering earlier tokens generated. We introduce CodeFusion, a pre-trained diffusion code generation model that addresses this limitation by iteratively denoising a complete program conditioned on the encoded natural language. We evaluate CodeFusion on the task of natural language to code generation for Bash, Python, and Microsoft Excel conditional formatting (CF) rules. Experiments show that CodeFusion (75M parameters) performs on par with state-of-the-art auto-regressive systems (350M-175B parameters) in top-1 accuracy and outperforms them in top-3 and top-5 accuracy due to its better balance in diversity versus quality.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 6 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

BERT Is Already a Denoiser: Cheaply Unlocking Iterative Generation in Masked Language Models

86% match2026

LeTI: Learning to Generate from Textual Interactions

86% match1 repo662023

Beyond Autoregression: An Empirical Study of Diffusion Large Language Models for Code Generation

85% match1 repo232025

Register Tokens for Bounded-State Reasoning in Diffusion Language Models

85% match2026

Noise Your Prompt: Noising Conditioning Tokens in Continuous Diffusion Language Models

85% match1 repo2026

AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation

84% match2 repos7462026

Cite this paper

@article{singh2023codefusion,
  title  = {CodeFusion: A Pre-trained Diffusion Model for Code Generation},
  author = {Mukul Singh and José Cambronero and Sumit Gulwani and Vu Le and Carina Negreanu and Gust Verbruggen},
  year   = {2023},
  eprint = {2310.17680},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2310.17680v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2310.17680.svg)](https://codeofpaper.com/paper/2310.17680)

Discussion