How Large Language Models are Transforming Machine-Paraphrased Plagiarism

Jan Philip Wahle, Terry Ruas, Frederic Kirstein, Bela Gipp
10/7/2022
cs.CLcs.AI

Official code: jpwahle/emnlp22-transforming (10 stars, last commit 2026-10-15).

Abstract

The recent success of large language models for text generation poses a severe threat to academic integrity, as plagiarists can generate realistic paraphrases indistinguishable from original work. However, the role of large autoregressive transformers in generating machine-paraphrased plagiarism and their detection is still developing in the literature. This work explores T5 and GPT-3 for machine-paraphrase generation on scientific articles from arXiv, student theses, and Wikipedia. We evaluate the detection performance of six automated solutions and one commercial plagiarism detection software and perform a human study with 105 participants regarding their detection performance and the quality of generated examples. Our results suggest that large models can rewrite text humans have difficulty identifying as machine-paraphrased (53% mean acc.). Human experts rate the quality of paraphrases generated by GPT-3 as high as original texts (clarity 4.0/5, fluency 4.2/5, coherence 3.8/5). The best-performing detection model (GPT-3) achieves a 66% F1-score in detecting paraphrases.

View on arXivView PDF

Code Implementations(1)

jpwahle/emnlp22-transformingOfficial100%

The official implementation of the EMNLP 2022 paper "How Large Language Models are Transforming Machine-Paraphrased Plagiarism".

100PythonOct 7, 20221 weeks ago
machine-learningnatural-language-processingnlpparaphrase-generationplagiarism

Similar Papers

Paraphrase Detection: Human vs. Machine Content

89% match1 repo32023

Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense

87% match1 repo2012023

PlagBench: Exploring the Duality of Large Language Models in Plagiarism Generation and Detection

87% match1 repo52024

Language Modeling and Understanding Through Paraphrase Generation and Detection

85% match2026

Mitigating Paraphrase Attacks on Machine-Text Detectors via Paraphrase Inversion

85% match2024

AI Content Self-Detection for Transformer-based Large Language Models

84% match2023

Cite this paper

@article{wahle2022large,
  title  = {How Large Language Models are Transforming Machine-Paraphrased Plagiarism},
  author = {Jan Philip Wahle and Terry Ruas and Frederic Kirstein and Bela Gipp},
  year   = {2022},
  eprint = {2210.03568},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2210.03568}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2210.03568.svg)](https://codeofpaper.com/paper/2210.03568)

Discussion