The Claire French Dialogue Dataset

Julie Hunter, Jérôme Louradour, Virgile Rennard, Ismaïl Harrando, Guokan Shang +1 more
11/28/2023
cs.CLcs.AI

No confident code implementation found for this paper.

Abstract

We present the Claire French Dialogue Dataset (CFDD), a resource created by members of LINAGORA Labs in the context of the OpenLLM France initiative. CFDD is a corpus containing roughly 160 million words from transcripts and stage plays in French that we have assembled and publicly released in an effort to further the development of multilingual, open source language models. This paper describes the 24 individual corpora of which CFDD is composed and provides links and citations to their original sources. It also provides our proposed breakdown of the full CFDD dataset into eight categories of subcorpora and describes the process we followed to standardize the format of the final dataset. We conclude with a discussion of similar work and future directions.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Carolina: a General Corpus of Contemporary Brazilian Portuguese with Provenance, Typology and Versioning Information

87% match2023

MC$^2$: Towards Transparent and Culturally-Aware NLP for Minority Languages in China

87% match1 repo372023

Common Corpus: The Largest Collection of Ethical Data for LLM Pre-Training

86% match2025

The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset

85% match2026

Skywork: A More Open Bilingual Foundation Model

85% match1 repo1.5k2023

Panda LLM: Training Data and Evaluation for Open-Sourced Chinese Instruction-Following Large Language Models

84% match1 repo1.0k2023

Cite this paper

@article{hunter2023claire,
  title  = {The Claire French Dialogue Dataset},
  author = {Julie Hunter and Jérôme Louradour and Virgile Rennard and Ismaïl Harrando and Guokan Shang and Jean-Pierre Lorré},
  year   = {2023},
  eprint = {2311.16840},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2311.16840v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2311.16840.svg)](https://codeofpaper.com/paper/2311.16840)

Discussion