TartuNLP @ SIGTYP 2024 Shared Task: Adapting XLM-RoBERTa for Ancient and Historical Languages

Aleksei Dorkin, Kairit Sirts
4/19/2024
cs.CL

No confident code implementation found for this paper.

Abstract

We present our submission to the unconstrained subtask of the SIGTYP 2024 Shared Task on Word Embedding Evaluation for Ancient and Historical Languages for morphological annotation, POS-tagging, lemmatization, character- and word-level gap-filling. We developed a simple, uniform, and computationally lightweight approach based on the adapters framework using parameter-efficient fine-tuning. We applied the same adapter-based approach uniformly to all tasks and 16 languages by fine-tuning stacked language- and task-specific adapters. Our submission obtained an overall second place out of three submissions, with the first place in word-level gap-filling. Our results show the feasibility of adapting language models pre-trained on modern languages to historical and ancient languages via adapter training.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Heidelberg-Boston @ SIGTYP 2024 Shared Task: Enhancing Low-Resource Language Analysis With Character-Aware Hierarchical Transformers

83% match1 repo12024

Under-resourced studies of under-resourced languages: lemmatization and POS-tagging with LLM annotators for historical Armenian, Georgian, Greek and Syriac

81% match2026

Adapters for Altering LLM Vocabularies: What Languages Benefit the Most?

81% match1 repo62026

Modern Models, Medieval Texts: A POS Tagging Study of Old Occitan

80% match2025

Ground Truth Generation for Multilingual Historical NLP using LLMs

80% match2025

Optimizing LLMs for Italian: Reducing Token Fertility and Enhancing Efficiency Through Vocabulary Adaptation

80% match1 repo42025

Cite this paper

@article{dorkin2024tartunlp,
  title  = {TartuNLP @ SIGTYP 2024 Shared Task: Adapting XLM-RoBERTa for Ancient and Historical Languages},
  author = {Aleksei Dorkin and Kairit Sirts},
  year   = {2024},
  eprint = {2404.12845},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2404.12845v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2404.12845.svg)](https://codeofpaper.com/paper/2404.12845)

Discussion