The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Models

Matthew Renze, Erhan Guven
1/11/2024
cs.CLcs.AI

Official code: matthewrenze/jhu-concise-cot (25 stars, BSD-2-Clause, last commit 2024-11-25).

Abstract

In this paper, we introduce Concise Chain-of-Thought (CCoT) prompting. We compared standard CoT and CCoT prompts to see how conciseness impacts response length and correct-answer accuracy. We evaluated this using GPT-3.5 and GPT-4 with a multiple-choice question-and-answer (MCQA) benchmark. CCoT reduced average response length by 48.70% for both GPT-3.5 and GPT-4 while having a negligible impact on problem-solving performance. However, on math problems, GPT-3.5 with CCoT incurs a performance penalty of 27.69%. Overall, CCoT leads to an average per-token cost reduction of 22.67%. All code, data, and supplemental materials are available on GitHub at https://github.com/matthewrenze/jhu-concise-cot

View on arXivView PDF

Code Implementations(1)

matthewrenze/jhu-concise-cotOfficial100%
BSD-2-Clause

The Benefits of a Concise Chain of Thought on Problem Solving in Large Language Models

253PythonDec 8, 20231 years agoBSD-2-Clause

Similar Papers

Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost

87% match2024

Prompting Science Report 2: The Decreasing Value of Chain of Thought in Prompting

87% match2025

Early Stopping Chain-of-thoughts in Large Language Models

85% match2025

To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning

85% match1 repo262024

Break the Chain: Large Language Models Can be Shortcut Reasoners

84% match2024

Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning

84% match1 repo232025

Cite this paper

@article{renze2024benefits,
  title  = {The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Models},
  author = {Matthew Renze and Erhan Guven},
  year   = {2024},
  eprint = {2401.05618},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2401.05618v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2401.05618.svg)](https://codeofpaper.com/paper/2401.05618)

Discussion