ParaNames 1.0: Creating an Entity Name Corpus for 400+ Languages using Wikidata

Jonne Sälevä, Constantine Lignos
5/15/2024
cs.CLcs.AI

No official code found. Best implementation: bltlab/paranames (42 stars, MIT, last commit 2024-05-20).

Abstract

We introduce ParaNames, a massively multilingual parallel name resource consisting of 140 million names spanning over 400 languages. Names are provided for 16.8 million entities, and each entity is mapped from a complex type hierarchy to a standard type (PER/LOC/ORG). Using Wikidata as a source, we create the largest resource of this type to date. We describe our approach to filtering and standardizing the data to provide the best quality possible. ParaNames is useful for multilingual language processing, both in defining tasks for name translation/transliteration and as supplementary data for tasks such as named entity recognition and linking. We demonstrate the usefulness of ParaNames on two tasks. First, we perform canonical name translation between English and 17 other languages. Second, we use it as a gazetteer for multilingual named entity recognition, obtaining performance improvements on all 10 languages evaluated.

View on arXivView PDF

Code Implementations(1)

bltlab/paranamesCommunity100%
MIT

ParaNames: A multilingual resource for parallel names

425Shell, PythonFeb 24, 20222 years agoMIT

Similar Papers

CebuaNER: A New Baseline Cebuano Named Entity Recognition Model

81% match1 repo52023

FiNERweb: Datasets and Artifacts for Scalable Multilingual Named Entity Recognition

81% match2025

Do LLMs Judge Distantly Supervised Named Entity Labels Well? Constructing the JudgeWEL Dataset

80% match2026

AWED-FiNER: Agents, Web applications, and Expert Detectors for Fine-grained Named Entity Recognition across 36 Languages for 6.6 Billion Speakers

80% match1 repo12026

DaMuEL: A Large Multilingual Dataset for Entity Linking

79% match2023

A Multi-way Parallel Named Entity Annotated Corpus for English, Tamil and Sinhala

79% match1 repo12024

Cite this paper

@article{slev2024paranames,
  title  = {ParaNames 1.0: Creating an Entity Name Corpus for 400+ Languages using Wikidata},
  author = {Jonne Sälevä and Constantine Lignos},
  year   = {2024},
  eprint = {2405.09496},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2405.09496v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2405.09496.svg)](https://codeofpaper.com/paper/2405.09496)

Discussion