Decoding Stumpers: Large Language Models vs. Human Problem-Solvers

Alon Goldstein, Miriam Havin, Roi Reichart, Ariel Goldstein
10/25/2023
cs.CLcs.HC

No confident code implementation found for this paper. 3 possible matches can be shown below.

Abstract

This paper investigates the problem-solving capabilities of Large Language Models (LLMs) by evaluating their performance on stumpers, unique single-step intuition problems that pose challenges for human solvers but are easily verifiable. We compare the performance of four state-of-the-art LLMs (Davinci-2, Davinci-3, GPT-3.5-Turbo, GPT-4) to human participants. Our findings reveal that the new-generation LLMs excel in solving stumpers and surpass human performance. However, humans exhibit superior skills in verifying solutions to the same problems. This research enhances our understanding of LLMs' cognitive abilities and provides insights for enhancing their problem-solving potential across various domains.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 3 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

NLPBench: Evaluating Large Language Models on Solving NLP Problems

85% match1 repo102023

Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models

85% match1 repo392023

Competition-Level Problems are Effective LLM Evaluators

85% match2023

VGRP-Bench: Visual Grid Reasoning Puzzle Benchmark for Large Vision-Language Models

84% match2025

GPT-3.5, GPT-4, or BARD? Evaluating LLMs Reasoning Ability in Zero-Shot Setting and Performance Boosting Through Prompts

84% match2023

MacGyver: Are Large Language Models Creative Problem Solvers?

84% match1 repo322023

Cite this paper

@article{goldstein2023decoding,
  title  = {Decoding Stumpers: Large Language Models vs. Human Problem-Solvers},
  author = {Alon Goldstein and Miriam Havin and Roi Reichart and Ariel Goldstein},
  year   = {2023},
  eprint = {2310.16411},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2310.16411v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2310.16411.svg)](https://codeofpaper.com/paper/2310.16411)

Discussion