Modeling Real-Time Interactive Conversations as Timed Diarized Transcripts

Garrett Tanzer, Gustaf Ahdritz, Luke Melas-Kyriazi
5/21/2024
cs.LGcs.CL

No official code found. Best implementation: gahdritz/rtic (0 stars, MIT, last commit 2024-06-03).

Abstract

Chatbots built upon language models have exploded in popularity, but they have largely been limited to synchronous, turn-by-turn dialogues. In this paper we present a simple yet general method to simulate real-time interactive conversations using pretrained text-only language models, by modeling timed diarized transcripts and decoding them with causal rejection sampling. We demonstrate the promise of this method with two case studies: instant messenger dialogues and spoken conversations, which require generation at about 30 tok/s and 20 tok/s respectively to maintain real-time interactivity. These capabilities can be added into language models using relatively little data and run on commodity hardware.

View on arXivView PDF

Code Implementations(1)

gahdritz/rticCommunity100%
MIT

Official code for the paper "Modeling Real-Time Interactive Conversations as Timed Diarized Transcripts"

00PythonMar 30, 20242 years agoMIT

Similar Papers

Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Models

84% match1 repo512024

PredGen: Accelerated Inference of Large Language Models through Input-Time Speculation for Real-Time Speech Interaction

83% match2025

Game-Time: Evaluating Temporal Dynamics in Spoken Language Models

83% match1 repo102025

Language Model Can Listen While Speaking

83% match2024

Real-Time Textless Dialogue Generation

83% match1 repo82025

Enabling Real-Time Conversations with Minimal Training Costs

82% match2024

Cite this paper

@article{tanzer2024modeling,
  title  = {Modeling Real-Time Interactive Conversations as Timed Diarized Transcripts},
  author = {Garrett Tanzer and Gustaf Ahdritz and Luke Melas-Kyriazi},
  year   = {2024},
  eprint = {2405.13203},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2405.13203v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2405.13203.svg)](https://codeofpaper.com/paper/2405.13203)

Discussion