Linear Classifier: An Often-Forgotten Baseline for Text Classification

Yu-Chen Lin, Si-An Chen, Jie-Jyun Liu, Chih-Jen Lin
6/12/2023
cs.CLcs.AIcs.LG

No official code found. Best implementation: JamesLYC88/text_classification_baseline_code (19 stars, last commit 2024-06-29).

Abstract

Large-scale pre-trained language models such as BERT are popular solutions for text classification. Due to the superior performance of these advanced methods, nowadays, people often directly train them for a few epochs and deploy the obtained model. In this opinion paper, we point out that this way may only sometimes get satisfactory results. We argue the importance of running a simple baseline like linear classifiers on bag-of-words features along with advanced methods. First, for many text data, linear methods show competitive performance, high efficiency, and robustness. Second, advanced models such as BERT may only achieve the best results if properly applied. Simple baselines help to confirm whether the results of advanced models are acceptable. Our experimental results fully support these points.

View on arXivView PDF

Code Implementations(1)

JamesLYC88/text_classification_baseline_codeCommunity100%

The code for the ACL 2023 paper "Linear Classifier: An Often-Forgotten Baseline for Text Classification".

192Shell, PythonJun 10, 20232 years ago
bertlarge-language-modelslinear-classifiersmachine-learningmulti-class-classification+5 more

Similar Papers

Advances in Pre-trained Language Models for Domain-Specific Text Classification: A Systematic Review

84% match2025

Text Classification in the LLM Era -- Where do we stand?

84% match2025

LLM-as-classifier: Semi-Supervised, Iterative Framework for Hierarchical Text Classification using Large Language Models

82% match2025

Augmenting Human-Annotated Training Data with Large Language Model Generation and Distillation in Open-Response Assessment

82% match2025

Language Models for Text Classification: Is In-Context Learning Enough?

82% match2024

Natural Language Processing Models for Robust Document Categorization

82% match2026

Cite this paper

@article{lin2023linear,
  title  = {Linear Classifier: An Often-Forgotten Baseline for Text Classification},
  author = {Yu-Chen Lin and Si-An Chen and Jie-Jyun Liu and Chih-Jen Lin},
  year   = {2023},
  eprint = {2306.07111},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2306.07111v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2306.07111.svg)](https://codeofpaper.com/paper/2306.07111)

Discussion