BERTweet: A pre-trained language model for English Tweets

Dat Quoc Nguyen, Thanh Vu, Anh Tuan Nguyen
5/20/2020
cs.CLcs.LG

Official code: VinAIResearch/BERTweet (610 stars, MIT, last commit 2024-07-22).

Abstract

We present BERTweet, the first public large-scale pre-trained language model for English Tweets. Our BERTweet, having the same architecture as BERT-base (Devlin et al., 2019), is trained using the RoBERTa pre-training procedure (Liu et al., 2019). Experiments show that BERTweet outperforms strong baselines RoBERTa-base and XLM-R-base (Conneau et al., 2020), producing better performance results than the previous state-of-the-art models on three Tweet NLP tasks: Part-of-speech tagging, Named-entity recognition and text classification. We release BERTweet under the MIT License to facilitate future research and applications on Tweet data. Our BERTweet is available at https://github.com/VinAIResearch/BERTweet

View on arXivView PDF

Code Implementations(1)

VinAIResearch/BERTweetOfficial100%
MIT

BERTweet: A pre-trained language model for English Tweets (EMNLP-2020)

61055PythonMay 6, 20202 years agoMIT
bertbertweetbertweet-covid19covidcovid-19+14 more

Similar Papers

GLM-130B: An Open Bilingual Pre-trained Model

88% match1 repo7.6k2022

How to Fine-Tune BERT for Text Classification?

87% match1 repo6402019

Towards Robust Named Entity Recognition for Historic German

87% match1 repo182019

Typhoon: Thai Large Language Models

87% match2023

DataComp-LM: In search of the next generation of training sets for language models

87% match2024

Unified Language Model Pre-training for Natural Language Understanding and Generation

87% match1 repo22.2k2019

Cite this paper

@article{nguyen2020bertweet,
  title  = {BERTweet: A pre-trained language model for English Tweets},
  author = {Dat Quoc Nguyen and Thanh Vu and Anh Tuan Nguyen},
  year   = {2020},
  eprint = {2005.10200},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2005.10200}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2005.10200.svg)](https://codeofpaper.com/paper/2005.10200)

Discussion