A benchmark for toxic comment classification on Civil Comments dataset

Corentin Duchene, Henri Jamet, Pierre Guillaume, Reda Dehak
1/26/2023
cs.CLeess.AS

No official code found. Best implementation: nigiva/hatespeech-detection-models (4 stars, last commit 2023-02-13).

Abstract

Toxic comment detection on social media has proven to be essential for content moderation. This paper compares a wide set of different models on a highly skewed multi-label hate speech dataset. We consider inference time and several metrics to measure performance and bias in our comparison. We show that all BERTs have similar performance regardless of the size, optimizations or language used to pre-train the models. RNNs are much faster at inference than any of the BERT. BiLSTM remains a good compromise between performance and inference time. RoBERTa with Focal Loss offers the best performance on biases and AUROC. However, DistilBERT combines both good AUROC and a low inference time. All models are affected by the bias of associating identities. BERT, RNN, and XLNet are less sensitive than the CNN and Compact Convolutional Transformers.

View on arXivView PDF

Code Implementations(1)

nigiva/hatespeech-detection-modelsCommunity100%

Model code for toxic comment detection

40Shell, PythonMar 24, 20223 years ago

Similar Papers

Deep Learning Approach for Classifying the Aggressive Comments on Social Media: Machine Translated Data Vs Real Life Data

86% match2023

Determination of toxic comments and unintended model bias minimization using Deep learning approach

85% match2023

Efficient Hate Speech Detection: Evaluating 38 Models from Traditional Methods to Transformers

84% match2025

A Comparative Study of PyCaret AutoML and CNN-BiLSTM for Binary Hate Speech Detection in Indonesian Twitter

84% match2026

Latent Feature-based Data Splits to Improve Generalisation Evaluation: A Hate Speech Detection Case Study

84% match2023

From Triage to Impact: A Multi-Hop Reasoning Benchmark for Explainable Persian Hate Speech

84% match1 repo2026

Cite this paper

@article{duchene2023benchmark,
  title  = {A benchmark for toxic comment classification on Civil Comments dataset},
  author = {Corentin Duchene and Henri Jamet and Pierre Guillaume and Reda Dehak},
  year   = {2023},
  eprint = {2301.11125},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2301.11125v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2301.11125.svg)](https://codeofpaper.com/paper/2301.11125)

Discussion