Cross-lingual Named Entity Corpus for Slavic Languages

Jakub Piskorski, Michał Marcińczuk, Roman Yangarber
3/30/2024
cs.CLcs.AIcs.LG

No official code found. Best implementation: SlavicNLP/SlavicNER (2 stars, last commit 2024-05-18).

Abstract

This paper presents a corpus manually annotated with named entities for six Slavic languages - Bulgarian, Czech, Polish, Slovenian, Russian, and Ukrainian. This work is the result of a series of shared tasks, conducted in 2017-2023 as a part of the Workshops on Slavic Natural Language Processing. The corpus consists of 5 017 documents on seven topics. The documents are annotated with five classes of named entities. Each entity is described by a category, a lemma, and a unique cross-lingual identifier. We provide two train-tune dataset splits - single topic out and cross topics. For each split, we set benchmarks using a transformer-based neural network architecture with the pre-trained multilingual models - XLM-RoBERTa-large for named entity mention recognition and categorization, and mT5-large for named entity lemmatization and linking.

View on arXivView PDF

Code Implementations(1)

SlavicNLP/SlavicNERCommunity100%

Cross-lingual Named Entity Corpus for Slavic Languages

20Mar 25, 20242 years ago

Similar Papers

Human-Annotated NER Dataset for the Kyrgyz Language

81% match2025

AlbNER: A Corpus for Named Entity Recognition in Albanian

80% match2023

MSNER: A Multilingual Speech Dataset for Named Entity Recognition

80% match2024

Evaluating Named Entity Recognition Models for Russian Cultural News Texts: From BERT to LLM

79% match2025

Revisiting Projection-based Data Transfer for Cross-Lingual Named Entity Recognition in Low-Resource Languages

79% match1 repo2025

A Multi-way Parallel Named Entity Annotated Corpus for English, Tamil and Sinhala

79% match1 repo12024

Cite this paper

@article{piskorski2024crosslingual,
  title  = {Cross-lingual Named Entity Corpus for Slavic Languages},
  author = {Jakub Piskorski and Michał Marcińczuk and Roman Yangarber},
  year   = {2024},
  eprint = {2404.00482},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2404.00482v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2404.00482.svg)](https://codeofpaper.com/paper/2404.00482)

Discussion