Vārta: A Large-Scale Headline-Generation Dataset for Indic Languages

Rahul Aralikatte, Ziling Cheng, Sumanth Doddapaneni, Jackie Chi Kit Cheung
5/10/2023
cs.CL

No official code found. Best implementation: rahular/varta (8 stars, Apache-2.0, last commit 2023-10-26).

Abstract

We present Vārta, a large-scale multilingual dataset for headline generation in Indic languages. This dataset includes 41.8 million news articles in 14 different Indic languages (and English), which come from a variety of high-quality sources. To the best of our knowledge, this is the largest collection of curated articles for Indic languages currently available. We use the data collected in a series of experiments to answer important questions related to Indic NLP and multilinguality research in general. We show that the dataset is challenging even for state-of-the-art abstractive models and that they perform only slightly better than extractive baselines. Owing to its size, we also show that the dataset can be used to pretrain strong language models that outperform competitive baselines in both NLU and NLG benchmarks.

View on arXivView PDF

Code Implementations(1)

rahular/vartaCommunity100%
Apache-2.0
80Shell, PythonMay 10, 20232 years agoApache-2.0

Similar Papers

Mukhyansh: A Headline Generation Dataset for Indic Languages

93% match2023

NLEBench+NorGLM: A Comprehensive Empirical Analysis and Benchmark Dataset for Generative Language Models in Norwegian

87% match1 repo22023

ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning

87% match2023

MasakhaNEWS: News Topic Classification for African languages

87% match1 repo262023

Multi-CrossRE A Multi-Lingual Multi-Domain Dataset for Relation Extraction

87% match1 repo492023

Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOM

87% match1 repo112023

Cite this paper

@article{aralikatte2023vrta,
  title  = {Vārta: A Large-Scale Headline-Generation Dataset for Indic Languages},
  author = {Rahul Aralikatte and Ziling Cheng and Sumanth Doddapaneni and Jackie Chi Kit Cheung},
  year   = {2023},
  eprint = {2305.05858},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2305.05858v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2305.05858.svg)](https://codeofpaper.com/paper/2305.05858)

Discussion