OLISIA: a Cascade System for Spoken Dialogue State Tracking

Léo Jacqmin, Lucas Druart, Yannick Estève, Benoît Favre, Lina Maria Rojas-Barahona +1 more
4/20/2023
eess.AScs.AIcs.CL

No official code found. Best implementation: Orange-OpenSource/olisia-dstc11 (9 stars, MIT, last commit 2023-07-06).

Abstract

Though Dialogue State Tracking (DST) is a core component of spoken dialogue systems, recent work on this task mostly deals with chat corpora, disregarding the discrepancies between spoken and written language.In this paper, we propose OLISIA, a cascade system which integrates an Automatic Speech Recognition (ASR) model and a DST model. We introduce several adaptations in the ASR and DST modules to improve integration and robustness to spoken conversations.With these adaptations, our system ranked first in DSTC11 Track 3, a benchmark to evaluate spoken DST. We conduct an in-depth analysis of the results and find that normalizing the ASR outputs and adapting the DST inputs through data augmentation, along with increasing the pre-trained models size all play an important role in reducing the performance discrepancy between written and spoken conversations.

View on arXivView PDF

Code Implementations(1)

Orange-OpenSource/olisia-dstc11Community100%
MIT

Training and evaluation of the OLISIA cascade system for Dialogue State Tracking (DST) on the MultiWoz dataset which ranked first in the "Speech Aware Dialog Systems Technology Challenge" track of the eleventh edition of the Dialogue Systems Technology Challenge

91Shell, PythonApr 17, 20233 years agoMIT

Similar Papers

Towards Joint Modeling of Dialogue Response and Speech Synthesis based on Large Language Model

86% match1 repo182023

SpokenWOZ: A Large-Scale Speech-Text Benchmark for Spoken Task-Oriented Dialogue Agents

85% match1 repo1.6k2023

Are cascade dialogue state tracking models speaking out of turn in spoken dialogues?

85% match2023

Scaling Conversational Hungarian ASR: The BEA-Dialogue+ Corpus

84% match2026

SpokenUS: A Spoken User Simulator for Task-Oriented Dialogue

84% match2026

Adapting Text-based Dialogue State Tracker for Spoken Dialogues

84% match2023

Cite this paper

@article{jacqmin2023olisia,
  title  = {OLISIA: a Cascade System for Spoken Dialogue State Tracking},
  author = {Léo Jacqmin and Lucas Druart and Yannick Estève and Benoît Favre and Lina Maria Rojas-Barahona and Valentin Vielzeuf},
  year   = {2023},
  eprint = {2304.11073},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2304.11073v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2304.11073.svg)](https://codeofpaper.com/paper/2304.11073)

Discussion