TookaBERT: A Step Forward for Persian NLU

MohammadAli SadraeiJavaheri, Ali Moghaddaszadeh, Milad Molazadeh, Fariba Naeiji, Farnaz Aghababaloo +5 more
7/23/2024
cs.CL

No confident code implementation found for this paper.

Abstract

The field of natural language processing (NLP) has seen remarkable advancements, thanks to the power of deep learning and foundation models. Language models, and specifically BERT, have been key players in this progress. In this study, we trained and introduced two new BERT models using Persian data. We put our models to the test, comparing them to seven existing models across 14 diverse Persian natural language understanding (NLU) tasks. The results speak for themselves: our larger model outperforms the competition, showing an average improvement of at least +2.8 points. This highlights the effectiveness and potential of our new BERT models for Persian NLU tasks.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

FaBERT: Pre-training BERT on Persian Blogs

88% match1 repo122024

Benchmarking Open-Source Large Language Models for Persian in Zero-Shot and Few-Shot Learning

85% match2025

Hakim: Farsi Text Embedding Model

84% match2025

FarSSiBERT: A Novel Transformer-based Model for Semantic Similarity Measurement of Persian Social Networks Informal Texts

84% match2024

Building a Rich Dataset to Empower the Persian Question Answering Systems

83% match2024

Fine-tuning Transformer-based Encoder for Turkish Language Understanding Tasks

82% match2024

Cite this paper

@article{sadraeijavaheri2024tookabert,
  title  = {TookaBERT: A Step Forward for Persian NLU},
  author = {MohammadAli SadraeiJavaheri and Ali Moghaddaszadeh and Milad Molazadeh and Fariba Naeiji and Farnaz Aghababaloo and Hamideh Rafiee and Zahra Amirmahani and Tohid Abedini and Fatemeh Zahra Sheikhi and Amirmohammad Salehoof},
  year   = {2024},
  eprint = {2407.16382},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.16382v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.16382.svg)](https://codeofpaper.com/paper/2407.16382)

Discussion