LUNA: A Framework for Language Understanding and Naturalness Assessment

Marat Saidov, Aleksandra Bakalova, Ekaterina Taktasheva, Vladislav Mikhailov, Ekaterina Artemova
1/9/2024
cs.CL

No official code found. Best implementation: Moonlight-Syntax/LUNA (12 stars, MIT, last commit 2023-09-09).

Abstract

The evaluation of Natural Language Generation (NLG) models has gained increased attention, urging the development of metrics that evaluate various aspects of generated text. LUNA addresses this challenge by introducing a unified interface for 20 NLG evaluation metrics. These metrics are categorized based on their reference-dependence and the type of text representation they employ, from string-based n-gram overlap to the utilization of static embeddings and pre-trained language models. The straightforward design of LUNA allows for easy extension with novel metrics, requiring just a few lines of code. LUNA offers a user-friendly tool for evaluating generated texts.

View on arXivView PDF

Code Implementations(1)

Moonlight-Syntax/LUNACommunity100%
MIT

LUNA: a Framework for Language Understanding and Naturalness Assessment.

121PythonSep 9, 20233 years agoMIT

Cite this paper

@article{saidov2024luna,
  title  = {LUNA: A Framework for Language Understanding and Naturalness Assessment},
  author = {Marat Saidov and Aleksandra Bakalova and Ekaterina Taktasheva and Vladislav Mikhailov and Ekaterina Artemova},
  year   = {2024},
  eprint = {2401.04522},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2401.04522v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2401.04522.svg)](https://codeofpaper.com/paper/2401.04522)

Discussion