The BS-meter: A ChatGPT-Trained Instrument to Detect Sloppy Language-Games

Alessandro Trevisan, Harry Giddens, Sarah Dillon, Alan F. Blackwell
11/22/2024
cs.CLcs.AIcs.HC

No confident code implementation found for this paper.

Abstract

What can we learn about language from studying how it is used by ChatGPT and other large language model (LLM)-based chatbots? In this paper, we analyse the distinctive character of language generated by ChatGPT, in relation to questions raised by natural language processing pioneer, and student of Wittgenstein, Margaret Masterman. Following frequent complaints that LLM-based chatbots produce "slop," or even "bullshit," in the sense of Frankfurt's popular monograph On Bullshit, we conduct an empirical study to contrast the language of 1,000 scientific publications with typical text generated by ChatGPT. We then explore whether the same language features can be detected in two well-known contexts of social dysfunction: George Orwell's critique of political speech, and David Graeber's characterisation of bullshit jobs. Using simple hypothesis-testing methods, we demonstrate that a statistical model of sloppy bullshit can reliably relate the Frankfurtian artificial bullshit of ChatGPT to the political and workplace functions of bullshit as observed in natural human language.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Machine Bullshit: Characterizing the Emergent Disregard for Truth in Large Language Models

85% match1 repo272025

A Linguistic Comparison between Human and ChatGPT-Generated Conversations

84% match1 repo32024

Empirical evidence of Large Language Model's influence on human spoken communication

83% match2024

Is English the New Programming Language? How About Pseudo-code Engineering?

83% match2024

ChatGPT is not A Man but Das Man: Representativeness and Structural Consistency of Silicon Samples Generated by Large Language Models

83% match2025

ChatGPT or A Silent Everywhere Helper: A Survey of Large Language Models

83% match2025

Cite this paper

@article{trevisan2024bsmeter,
  title  = {The BS-meter: A ChatGPT-Trained Instrument to Detect Sloppy Language-Games},
  author = {Alessandro Trevisan and Harry Giddens and Sarah Dillon and Alan F. Blackwell},
  year   = {2024},
  eprint = {2411.15129},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2411.15129v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2411.15129.svg)](https://codeofpaper.com/paper/2411.15129)

Discussion