Discourse Structure Extraction from Pre-Trained and Fine-Tuned Language Models in Dialogues

Chuyuan Li, Patrick Huber, Wen Xiao, Maxime Amblard, Chloé Braud +1 more
2/12/2023
cs.CL

No official code found. Best implementation: chuyuanli/PLM4DiscStruct (3 stars, last commit 2024-12-16).

Abstract

Discourse processing suffers from data sparsity, especially for dialogues. As a result, we explore approaches to build discourse structures for dialogues, based on attention matrices from Pre-trained Language Models (PLMs). We investigate multiple tasks for fine-tuning and show that the dialogue-tailored Sentence Ordering task performs best. To locate and exploit discourse information in PLMs, we propose an unsupervised and a semi-supervised method. Our proposals achieve encouraging results on the STAC corpus, with F1 scores of 57.2 and 59.3 for unsupervised and semi-supervised methods, respectively. When restricted to projective trees, our scores improved to 63.3 and 68.1.

View on arXivView PDF

Code Implementations(1)

chuyuanli/PLM4DiscStructCommunity64%

Source code for paper Discourse Structure Extraction from Pre-Trained and Fine-Tuned Language Models in Dialogues, EACL 2023 Findings

30Jupyter NotebookOct 18, 20231 years ago

Similar Papers

Uncovering the Potential of ChatGPT for Discourse Analysis in Dialogue: An Empirical Study

85% match1 repo42023

Cross-Genre Argument Mining: Can Language Models Automatically Fill in Missing Discourse Markers?

84% match2023

Multi-Stage Coarse-to-Fine Contrastive Learning for Conversation Intent Induction

83% match2023

Stabilized In-Context Learning with Pre-trained Language Models for Few Shot Dialogue State Tracking

83% match2023

Channel-aware Decoupling Network for Multi-turn Dialogue Comprehension

83% match2023

Hierarchical Dialogue Understanding with Special Tokens and Turn-level Attention

82% match1 repo192023

Cite this paper

@article{li2023discourse,
  title  = {Discourse Structure Extraction from Pre-Trained and Fine-Tuned Language Models in Dialogues},
  author = {Chuyuan Li and Patrick Huber and Wen Xiao and Maxime Amblard and Chloé Braud and Giuseppe Carenini},
  year   = {2023},
  eprint = {2302.05895},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2302.05895v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2302.05895.svg)](https://codeofpaper.com/paper/2302.05895)

Discussion