factgenie: A Framework for Span-based Evaluation of Generated Texts

Zdeněk Kasner, Ondřej Plátek, Patrícia Schmidtová, Simone Balloccu, Ondřej Dušek
7/25/2024
cs.CL

No official code found. Best implementation: ufal/factgenie (44 stars, MIT, last commit 2026-04-20).

Abstract

We present factgenie: a framework for annotating and visualizing word spans in textual model outputs. Annotations can capture various span-based phenomena such as semantic inaccuracies or irrelevant text. With factgenie, the annotations can be collected both from human crowdworkers and large language models. Our framework consists of a web interface for data visualization and gathering text annotations, powered by an easily extensible codebase.

View on arXivView PDF

Code Implementations(1)

ufal/factgenieCommunity100%
MIT

Lightweight self-hosted span annotation tool

448CSS, HTMLMay 16, 20246 months agoMIT
annotationannotation-toolannotationsdata-annotationllm+5 more

Cite this paper

@article{kasner2024factgenie,
  title  = {factgenie: A Framework for Span-based Evaluation of Generated Texts},
  author = {Zdeněk Kasner and Ondřej Plátek and Patrícia Schmidtová and Simone Balloccu and Ondřej Dušek},
  year   = {2024},
  eprint = {2407.17863},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.17863v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.17863.svg)](https://codeofpaper.com/paper/2407.17863)

Discussion