Does GPT-4 pass the Turing test?

Cameron R. Jones, Benjamin K. Bergen
10/31/2023
cs.AIcs.CL

No confident code implementation found for this paper.

Abstract

We evaluated GPT-4 in a public online Turing test. The best-performing GPT-4 prompt passed in 49.7% of games, outperforming ELIZA (22%) and GPT-3.5 (20%), but falling short of the baseline set by human participants (66%). Participants' decisions were based mainly on linguistic style (35%) and socioemotional traits (27%), supporting the idea that intelligence, narrowly conceived, is not sufficient to pass the Turing test. Participant knowledge about LLMs and number of games played positively correlated with accuracy in detecting AI, suggesting learning and practice as possible strategies to mitigate deception. Despite known limitations as a test of intelligence, we argue that the Turing test continues to be relevant as an assessment of naturalistic communication and deception. AI models with the ability to masquerade as humans could have widespread societal consequences, and we analyse the effectiveness of different strategies and criteria for judging humanlikeness.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Cite this paper

@article{jones2023does,
  title  = {Does GPT-4 pass the Turing test?},
  author = {Cameron R. Jones and Benjamin K. Bergen},
  year   = {2023},
  eprint = {2310.20216},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2310.20216v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2310.20216.svg)](https://codeofpaper.com/paper/2310.20216)

Discussion