Kakugo: Distillation of Low-Resource Languages into Small Language Models

Peter Devine, Mardhiyah Sanni, Farid Adilazuarda, Julieta Gil Loizaga, Barry Haddow
1/20/2026
cs.CLcs.AIcs.LG

No confident code implementation found for this paper. 4 possible matches can be shown below.

Abstract

We present Kakugo, a novel and cost-effective pipeline designed to train general-purpose Small Language Models (SLMs) for low-resource languages using only the language name as input. By using a large teacher model to generate synthetic prompts and translate instruction datasets, we produced training data and SLMs for 54 low-resource languages. Evaluations across a diverse set of general natural language processing tasks, including translation, classification, and question answering, demonstrate that our pipeline consistently improves performance over base models. With a total generation and training cost of under $50 per language, Kakugo offers an accessible method for communities to develop language-specific AI.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 4 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Better as Generators Than Classifiers: Leveraging LLMs and Synthetic Data for Low-Resource Multilingual Classification

88% match2026

Foundation Models for Low-Resource Language Education (Vision Paper)

86% match2024

MiniLingua: A Small Open-Source LLM for European Languages

85% match2025

Is Small Language Model the Silver Bullet to Low-Resource Languages Machine Translation?

84% match2025

Small Language Models: Survey, Measurements, and Insights

84% match1 repo1092024

TinyHelen's First Curriculum: Training and Evaluating Tiny Language Models in a Simpler Language Environment

84% match1 repo152024

Cite this paper

@article{devine2026kakugo,
  title  = {Kakugo: Distillation of Low-Resource Languages into Small Language Models},
  author = {Peter Devine and Mardhiyah Sanni and Farid Adilazuarda and Julieta Gil Loizaga and Barry Haddow},
  year   = {2026},
  eprint = {2601.14051},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2601.14051v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2601.14051.svg)](https://codeofpaper.com/paper/2601.14051)

Discussion