DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation

Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett +4 more
11/1/2019
cs.CLcs.LG

Official code: microsoft/DialoGPT (2,422 stars, MIT, PyTorch, last commit 2022-10-17).

Abstract

We present a large, tunable neural conversational response generation model, DialoGPT (dialogue generative pre-trained transformer). Trained on 147M conversation-like exchanges extracted from Reddit comment chains over a period spanning from 2005 through 2017, DialoGPT extends the Hugging Face PyTorch transformer to attain a performance close to human both in terms of automatic and human evaluation in single-turn dialogue settings. We show that conversational systems that leverage DialoGPT generate more relevant, contentful and context-consistent responses than strong baseline systems. The pre-trained model and training pipeline are publicly released to facilitate research into neural response generation and the development of more intelligent open-domain dialogue systems.

View on arXivView PDF

Code Implementations(1)

microsoft/DialoGPTOfficial100%
PyTorchMIT

Large-scale pretraining for dialogue

2,422346Shell, PythonAug 29, 20194 years agoMIT
data-processingdialogptdialoguegpt-2machine-learning+4 more

Similar Papers

PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generation

89% match1 repo6692021

PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable

88% match1 repo1.8k2019

ChatGPT: Applications, Opportunities, and Threats

87% match2023

Improving Neural Conversational Models with Entropy-Based Data Filtering

85% match2 repos972019

PICK: Polished & Informed Candidate Scoring for Knowledge-Grounded Dialogue Systems

84% match1 repo22023

Response Generation in Longitudinal Dialogues: Which Knowledge Representation Helps?

84% match2023

Cite this paper

@article{zhang2019dialogpt,
  title  = {DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation},
  author = {Yizhe Zhang and Siqi Sun and Michel Galley and Yen-Chun Chen and Chris Brockett and Xiang Gao and Jianfeng Gao and Jingjing Liu and Bill Dolan},
  year   = {2019},
  eprint = {1911.00536},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1911.00536}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1911.00536.svg)](https://codeofpaper.com/paper/1911.00536)

Discussion