Unsupervised End-to-End Task-Oriented Dialogue with LLMs: The Power of the Noisy Channel

Brendan King, Jeffrey Flanigan
4/23/2024
cs.CL

No official code found. Best implementation: jlab-nlp/nc_latent_tod (3 stars, MIT, last commit 2024-11-21).

Abstract

Training task-oriented dialogue systems typically requires turn-level annotations for interacting with their APIs: e.g. a dialogue state and the system actions taken at each step. These annotations can be costly to produce, error-prone, and require both domain and annotation expertise. With advances in LLMs, we hypothesize that unlabeled data and a schema definition are sufficient for building a working task-oriented dialogue system, completely unsupervised. We consider a novel unsupervised setting of only (1) a well-defined API schema (2) a set of unlabeled dialogues between a user and agent. We propose an innovative approach using expectation-maximization (EM) that infers turn-level annotations as latent variables using a noisy channel model to build an end-to-end dialogue agent. Evaluating our approach on the MultiWOZ benchmark, our method more than doubles the dialogue success rate of a strong GPT-3.5 baseline.

View on arXivView PDF

Code Implementations(1)

jlab-nlp/nc_latent_todCommunity100%
MIT

Unsupervised End-to-End Task-Oriented Dialogue with LLMs (EMNLP, 2024)

30Shell, PythonApr 23, 20241 years agoMIT

Similar Papers

Training Zero-Shot Generalizable End-to-End Task-Oriented Dialog System Without Turn-level Dialog Annotations

85% match2024

Evaluating and Enhancing Out-of-Domain Generalization of Task-Oriented Dialog Systems for Task Completion without Turn-level Dialog Annotations

84% match2025

Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk

82% match2024

Enhancing Dialogue State Tracking Models through LLM-backed User-Agents Simulation

82% match2024

Investigating Low-Cost LLM Annotation for~Spoken Dialogue Understanding Datasets

82% match2024

Improving Dialogue Agents by Decomposing One Global Explicit Annotation with Local Implicit Multimodal Feedback

82% match2024

Cite this paper

@article{king2024unsupervised,
  title  = {Unsupervised End-to-End Task-Oriented Dialogue with LLMs: The Power of the Noisy Channel},
  author = {Brendan King and Jeffrey Flanigan},
  year   = {2024},
  eprint = {2404.15219},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2404.15219v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2404.15219.svg)](https://codeofpaper.com/paper/2404.15219)

Discussion