NorBench -- A Benchmark for Norwegian Language Models

David Samuel, Andrey Kutuzov, Samia Touileb, Erik Velldal, Lilja Øvrelid +3 more
5/6/2023
cs.CL

No official code found. Best implementation: ltgoslo/norbench (14 stars, MIT, last commit 2025-08-29).

Abstract

We present NorBench: a streamlined suite of NLP tasks and probes for evaluating Norwegian language models (LMs) on standardized data splits and evaluation metrics. We also introduce a range of new Norwegian language models (both encoder and encoder-decoder based). Finally, we compare and analyze their performance, along with other existing LMs, across the different benchmark tests of NorBench.

View on arXivView PDF

Code Implementations(1)

ltgoslo/norbenchCommunity100%
MIT

Natural language understanding benchmarks for Norwegian

143Shell, PythonJun 24, 20211 years agoMIT

Similar Papers

Catwalk: A Unified Language Model Evaluation Framework for Many Datasets

88% match2 repos2.8k2023

BLUEX: A benchmark based on Brazilian Leading Universities Entrance eXams

87% match1 repo202023

DaG LLM ver 1.0: Pioneering Instruction-Tuned Language Modeling for Korean NLP

87% match2023

A Family of Pretrained Transformer Language Models for Russian

86% match2023

Advancing the Evaluation of Traditional Chinese Language Models: Towards a Comprehensive Benchmark Suite

86% match1 repo2912023

KyrgyzLLM-Bench: Benchmarking Kyrgyz Language Understanding

86% match2026

Cite this paper

@article{samuel2023norbench,
  title  = {NorBench -- A Benchmark for Norwegian Language Models},
  author = {David Samuel and Andrey Kutuzov and Samia Touileb and Erik Velldal and Lilja Øvrelid and Egil Rønningstad and Elina Sigdel and Anna Palatkina},
  year   = {2023},
  eprint = {2305.03880},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2305.03880v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2305.03880.svg)](https://codeofpaper.com/paper/2305.03880)

Discussion