A Comparison of Large Language Model and Human Performance on Random Number Generation Tasks

Rachel M. Harrison
8/19/2024
cs.AIcs.CLq-bio.NC

No official code found. Best implementation: paxnea/genAI-rngt (0 stars, MIT, last commit 2024-08-20).

Abstract

Random Number Generation Tasks (RNGTs) are used in psychology for examining how humans generate sequences devoid of predictable patterns. By adapting an existing human RNGT for an LLM-compatible environment, this preliminary study tests whether ChatGPT-3.5, a large language model (LLM) trained on human-generated text, exhibits human-like cognitive biases when generating random number sequences. Initial findings indicate that ChatGPT-3.5 more effectively avoids repetitive and sequential patterns compared to humans, with notably lower repeat frequencies and adjacent number frequencies. Continued research into different models, parameters, and prompting methodologies will deepen our understanding of how LLMs can more closely mimic human random generation behaviors, while also broadening their applications in cognitive and behavioral science research.

View on arXivView PDF

Code Implementations(1)

paxnea/genAI-rngtCommunity100%
MIT

A study of LLM performance on Random Number Generation Tasks (RNGTs)

00Jupyter NotebookJun 24, 20242 years agoMIT
chatgptgenerative-airng

Similar Papers

Deterministic or probabilistic? The psychology of LLMs as random number generators

90% match2025

Towards a Psychology of Machines: Large Language Models Predict Human Memory

85% match2024

In-Context Learning Dynamics with Random Binary Sequences

84% match2026

Humanlike Cognitive Patterns as Emergent Phenomena in Large Language Models

84% match2024

Aspects of human memory and Large Language Models

84% match1 repo32023

Language Models Trained to do Arithmetic Predict Human Risky and Intertemporal Choice

83% match2026

Cite this paper

@article{harrison2024comparison,
  title  = {A Comparison of Large Language Model and Human Performance on Random Number Generation Tasks},
  author = {Rachel M. Harrison},
  year   = {2024},
  eprint = {2408.09656},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2408.09656v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2408.09656.svg)](https://codeofpaper.com/paper/2408.09656)

Discussion