Fine-tuning Pre-trained Named Entity Recognition Models For Indian Languages

Sankalp Bahad, Pruthwik Mishra, Karunesh Arora, Rakesh Chandra Balabantaray, Dipti Misra Sharma +1 more
5/8/2024
cs.CL

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

Named Entity Recognition (NER) is a useful component in Natural Language Processing (NLP) applications. It is used in various tasks such as Machine Translation, Summarization, Information Retrieval, and Question-Answering systems. The research on NER is centered around English and some other major languages, whereas limited attention has been given to Indian languages. We analyze the challenges and propose techniques that can be tailored for Multilingual Named Entity Recognition for Indian Languages. We present a human annotated named entity corpora of 40K sentences for 4 Indian languages from two of the major Indian language families. Additionally,we present a multilingual model fine-tuned on our dataset, which achieves an F1 score of 0.80 on our dataset on average. We achieve comparable performance on completely unseen benchmark datasets for Indian languages which affirms the usability of our model.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

TriNER: A Series of Named Entity Recognition Models For Hindi, Bengali & Marathi

93% match2025

A Multi-way Parallel Named Entity Annotated Corpus for English, Tamil and Sinhala

91% match1 repo12024

Enhancing Low Resource NER Using Assisting Language And Transfer Learning

91% match2023

ANCHOLIK-NER: A Benchmark Dataset for Bangla Regional Named Entity Recognition

89% match1 repo12025

Enhancing NER Performance in Low-Resource Pakistani Languages using Cross-Lingual Data Augmentation

88% match2025

Gazetteer-Enhanced Bangla Named Entity Recognition with BanglaBERT Semantic Embeddings K-Means-Infused CRF Model

88% match2024

Cite this paper

@article{bahad2024finetuning,
  title  = {Fine-tuning Pre-trained Named Entity Recognition Models For Indian Languages},
  author = {Sankalp Bahad and Pruthwik Mishra and Karunesh Arora and Rakesh Chandra Balabantaray and Dipti Misra Sharma and Parameswari Krishnamurthy},
  year   = {2024},
  eprint = {2405.04829},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2405.04829v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2405.04829.svg)](https://codeofpaper.com/paper/2405.04829)

Discussion