NLP-ADBench: NLP Anomaly Detection Benchmark

Yuangang Li, Jiaqi Li, Zhuo Xiao, Tiankai Yang, Yi Nian +2 more
12/6/2024
cs.CLcs.LG

Official code: USC-FORTIS/NLP-ADBench (21 stars, MIT, last commit 2025-10-09).

Abstract

Anomaly detection (AD) is an important machine learning task with applications in fraud detection, content moderation, and user behavior analysis. However, AD is relatively understudied in a natural language processing (NLP) context, limiting its effectiveness in detecting harmful content, phishing attempts, and spam reviews. We introduce NLP-ADBench, the most comprehensive NLP anomaly detection (NLP-AD) benchmark to date, which includes eight curated datasets and 19 state-of-the-art algorithms. These span 3 end-to-end methods and 16 two-step approaches that adapt classical, non-AD methods to language embeddings from BERT and OpenAI. Our empirical results show that no single model dominates across all datasets, indicating a need for automated model selection. Moreover, two-step methods with transformer-based embeddings consistently outperform specialized end-to-end approaches, with OpenAI embeddings outperforming those of BERT. We release NLP-ADBench at https://github.com/USC-FORTIS/NLP-ADBench, providing a unified framework for NLP-AD and supporting future investigations.

View on arXivView PDF

Code Implementations(1)

USC-FORTIS/NLP-ADBenchOfficial100%
MIT

[EMNLP Findings 2025]. NLP-ADBench is a comprehensive benchmarking tool designed for Anomaly Detection in Natural Language Processing (NLP).

212PythonNov 28, 20241 years agoMIT
anomalyanomaly-detectionnlp

Similar Papers

Text-ADBench: Text Anomaly Detection Benchmark based on LLMs Embedding

89% match1 repo12025

AD-LLM: Benchmarking Large Language Models for Anomaly Detection

89% match1 repo442024

TAD-Bench: A Comprehensive Benchmark for Embedding-Based Text Anomaly Detection

84% match2025

LLM as an Algorithmist: Enhancing Anomaly Detectors via Programmatic Synthesis

80% match2025

NNG-Mix: Improving Semi-supervised Anomaly Detection with Pseudo-anomaly Generation

80% match1 repo192023

Are Anomaly Scores Telling the Whole Story? A Benchmark for Multilevel Anomaly Detection

80% match2024

Cite this paper

@article{li2024nlpadbench,
  title  = {NLP-ADBench: NLP Anomaly Detection Benchmark},
  author = {Yuangang Li and Jiaqi Li and Zhuo Xiao and Tiankai Yang and Yi Nian and Xiyang Hu and Yue Zhao},
  year   = {2024},
  eprint = {2412.04784},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2412.04784v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2412.04784.svg)](https://codeofpaper.com/paper/2412.04784)

Discussion