Assessing the Human Likeness of AI-Generated Counterspeech

Xiaoying Song, Sujana Mamidisetty, Eduardo Blanco, Lingzi Hong
10/14/2024
cs.CL

No official code found. Best implementation: oliveeeee25/counterspeech_eval_humanlike (4 stars, last commit 2024-12-15).

Abstract

Counterspeech is a targeted response to counteract and challenge abusive or hateful content. It effectively curbs the spread of hatred and fosters constructive online communication. Previous studies have proposed different strategies for automatically generated counterspeech. Evaluations, however, focus on relevance, surface form, and other shallow linguistic characteristics. This paper investigates the human likeness of AI-generated counterspeech, a critical factor influencing effectiveness. We implement and evaluate several LLM-based generation strategies, and discover that AI-generated and human-written counterspeech can be easily distinguished by both simple classifiers and humans. Further, we reveal differences in linguistic characteristics, politeness, and specificity. The dataset used in this study is publicly available for further research.

View on arXivView PDF

Code Implementations(1)

oliveeeee25/counterspeech_eval_humanlikeCommunity100%
40Dec 6, 20241 years ago

Similar Papers

CSEval: Towards Automated, Multi-Dimensional, and Reference-Free Counterspeech Evaluation using Auto-Calibrated LLMs

87% match2025

Outcome-Constrained Large Language Models for Countering Hate Speech

86% match2024

Is Safer Better? The Impact of Guardrails on the Argumentative Strength of LLMs in Hate Speech Countering

85% match1 repo22024

Hatred Stems from Ignorance! Distillation of the Persuasion Modes in Countering Conversational Hate Speech

85% match2024

Debunking with Dialogue? Exploring AI-Generated Counterspeech to Challenge Conspiracy Theories

84% match2025

Think Like a Person Before Responding: A Multi-Faceted Evaluation of Persona-Guided LLMs for Countering Hate

83% match1 repo22025

Cite this paper

@article{song2024assessing,
  title  = {Assessing the Human Likeness of AI-Generated Counterspeech},
  author = {Xiaoying Song and Sujana Mamidisetty and Eduardo Blanco and Lingzi Hong},
  year   = {2024},
  eprint = {2410.11007},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2410.11007v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2410.11007.svg)](https://codeofpaper.com/paper/2410.11007)

Discussion