Taxi1500: A Multilingual Dataset for Text Classification in 1500 Languages

Chunlan Ma, Ayyoob ImaniGooghari, Haotian Ye, Renhao Pei, Ehsaneddin Asgari +1 more
5/15/2023
cs.CL

No official code found. Best implementation: cisnlp/Taxi1500 (12 stars, Apache-2.0, last commit 2024-05-31).

Abstract

While natural language processing tools have been developed extensively for some of the world's languages, a significant portion of the world's over 7000 languages are still neglected. One reason for this is that evaluation datasets do not yet cover a wide range of languages, including low-resource and endangered ones. We aim to address this issue by creating a text classification dataset encompassing a large number of languages, many of which currently have little to no annotated data available. We leverage parallel translations of the Bible to construct such a dataset by first developing applicable topics and employing a crowdsourcing tool to collect annotated data. By annotating the English side of the data and projecting the labels onto other languages through aligned verses, we generate text classification datasets for more than 1500 languages. We extensively benchmark several existing multilingual language models using our dataset. To facilitate the advancement of research in this area, we will release our dataset and code.

View on arXivView PDF

Code Implementations(1)

cisnlp/Taxi1500Community100%
Apache-2.0

[NAACL 2025] Taxi1500: A Multilingual Dataset for Text Classification in 1500 Languages

120PythonJun 27, 20232 years agoApache-2.0

Similar Papers

SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects

89% match1 repo282023

CreoleVal: Multilingual Multitask Benchmarks for Creoles

87% match1 repo182023

CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages

87% match2023

BhashaSutra: A Task-Centric Unified Survey of Indian NLP Datasets, Corpora, and Resources

87% match2026

Breaking Language Barriers: A Question Answering Dataset for Hindi and Marathi

86% match2023

PersianLLaMA: Towards Building First Persian Large Language Model

86% match2023

Cite this paper

@article{ma2023taxi,
  title  = {Taxi1500: A Multilingual Dataset for Text Classification in 1500 Languages},
  author = {Chunlan Ma and Ayyoob ImaniGooghari and Haotian Ye and Renhao Pei and Ehsaneddin Asgari and Hinrich Schütze},
  year   = {2023},
  eprint = {2305.08487},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2305.08487v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2305.08487.svg)](https://codeofpaper.com/paper/2305.08487)

Discussion