Self-Reflection in LLM Agents: Effects on Problem-Solving Performance

Matthew Renze, Erhan Guven
5/5/2024
cs.CLcs.AI

Official code: matthewrenze/self-reflection (97 stars, BSD-2-Clause, last commit 2024-11-25).

Abstract

In this study, we investigated the effects of self-reflection in large language models (LLMs) on problem-solving performance. We instructed nine popular LLMs to answer a series of multiple-choice questions to provide a performance baseline. For each incorrectly answered question, we instructed eight types of self-reflecting LLM agents to reflect on their mistakes and provide themselves with guidance to improve problem-solving. Then, using this guidance, each self-reflecting agent attempted to re-answer the same questions. Our results indicate that LLM agents are able to significantly improve their problem-solving performance through self-reflection ($p < 0.001$). In addition, we compared the various types of self-reflection to determine their individual contribution to performance. All code and data are available on GitHub at https://github.com/matthewrenze/self-reflection

View on arXivView PDF

Code Implementations(1)

matthewrenze/self-reflectionOfficial100%
BSD-2-Clause

Self-Reflection in LLM Agents: Effects on Problem-Solving Performance

9710PythonMay 2, 20241 years agoBSD-2-Clause

Similar Papers

Self-Reflection Makes Large Language Models Safer, Less Biased, and Ideologically Neutral

88% match2024

Reflect, Retry, Reward: Self-Improving LLMs via Reinforcement Learning

88% match2025

Self-Contrast: Better Reflection Through Inconsistent Solving Perspectives

87% match2024

SAMULE: Self-Learning Agents Enhanced by Multi-level Reflection

86% match2025

From Emergence to Control: Probing and Modulating Self-Reflection in Language Models

85% match1 repo32025

Self-Correcting Large Language Models: Generation vs. Multiple Choice

84% match2025

Cite this paper

@article{renze2024selfreflection,
  title  = {Self-Reflection in LLM Agents: Effects on Problem-Solving Performance},
  author = {Matthew Renze and Erhan Guven},
  year   = {2024},
  eprint = {2405.06682},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2405.06682v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2405.06682.svg)](https://codeofpaper.com/paper/2405.06682)

Discussion