Benchmarking Abstractive Summarisation: A Dataset of Human-authored Summaries of Norwegian News Articles

Samia Touileb, Vladislav Mikhailov, Marie Kroka, Lilja Øvrelid, Erik Velldal
1/13/2025
cs.CL

No confident code implementation found for this paper.

Abstract

We introduce a dataset of high-quality human-authored summaries of news articles in Norwegian. The dataset is intended for benchmarking the abstractive summarisation capabilities of generative language models. Each document in the dataset is provided with three different candidate gold-standard summaries written by native Norwegian speakers, and all summaries are provided in both of the written variants of Norwegian -- Bokmål and Nynorsk. The paper describes details on the data creation effort as well as an evaluation of existing open LLMs for Norwegian on the dataset. We also provide insights from a manual human evaluation, comparing human-authored to model-generated summaries. Our results indicate that the dataset provides a challenging LLM benchmark for Norwegian summarisation capabilities

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Event-based evaluation of abstractive news summarization

85% match2025

Unraveling the Capabilities of Language Models in News Summarization

83% match1 repo22025

Summarization Metrics for Spanish and Basque: Do Automatic Scores and LLM-Judges Correlate with Humans?

83% match2025

Benchmarking Large Language Models for News Summarization

83% match1 repo472023

Evaluating LLMs and Pre-trained Models for Text Summarization Across Diverse Datasets

82% match2025

DomainSum: A Hierarchical Benchmark for Fine-Grained Domain Shift in Abstractive Text Summarization

82% match1 repo22024

Cite this paper

@article{touileb2025benchmarking,
  title  = {Benchmarking Abstractive Summarisation: A Dataset of Human-authored Summaries of Norwegian News Articles},
  author = {Samia Touileb and Vladislav Mikhailov and Marie Kroka and Lilja Øvrelid and Erik Velldal},
  year   = {2025},
  eprint = {2501.07718},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2501.07718v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2501.07718.svg)](https://codeofpaper.com/paper/2501.07718)

Discussion