ETR: Entropy Trend Reward for Efficient Chain-of-Thought Reasoning

Xuan Xiong, Huan Liu, Li Gu, Zhixiang Chi, Yue Qiu +2 more
4/7/2026
cs.AIcs.CL

Official code: Xuan1030/ETR (10 stars, last commit 2026-04-18).

Abstract

Chain-of-thought (CoT) reasoning improves large language model performance on complex tasks, but often produces excessively long and inefficient reasoning traces. Existing methods shorten CoTs using length penalties or global entropy reduction, implicitly assuming that low uncertainty is desirable throughout reasoning. We show instead that reasoning efficiency is governed by the trajectory of uncertainty. CoTs with dominant downward entropy trends are substantially shorter. Motivated by this insight, we propose Entropy Trend Reward (ETR), a trajectory-aware objective that encourages progressive uncertainty reduction while allowing limited local exploration. We integrate ETR into Group Relative Policy Optimization (GRPO) and evaluate it across multiple reasoning models and challenging benchmarks. ETR consistently achieves a superior accuracy-efficiency tradeoff, improving DeepSeek-R1-Distill-7B by 9.9% in accuracy while reducing CoT length by 67% across four benchmarks. Code is available at https://github.com/Xuan1030/ETR

View on arXivView PDF

Code Implementations(1)

Xuan1030/ETROfficial100%

ETR: Entropy Trend Reward for Efficient Chain-of-Thought Reasoning

100Shell, PythonDec 31, 20256 months ago

Similar Papers

Learning When to Sample: Confidence-Aware Self-Consistency for Efficient LLM Chain-of-Thought Reasoning

88% match2026

SmartThinker: Progressive Chain-of-Thought Length Calibration for Efficient Large Language Model Reasoning

87% match1 repo162026

Improving reasoning at inference time via uncertainty minimisation

87% match2026

Think or Not? Exploring Thinking Efficiency in Large Reasoning Models via an Information-Theoretic Lens

87% match2025

Entropy trajectory shape predicts LLM reasoning reliability: A diagnostic study of uncertainty dynamics in chain-of-thought

86% match2026

Answer-Distribution Trajectories: A Stochastic-Dynamics View of LLM Reasoning

86% match2026

Cite this paper

@article{xiong2026etr,
  title  = {ETR: Entropy Trend Reward for Efficient Chain-of-Thought Reasoning},
  author = {Xuan Xiong and Huan Liu and Li Gu and Zhixiang Chi and Yue Qiu and Yuanhao Yu and Yang Wang},
  year   = {2026},
  eprint = {2604.05355},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2604.05355v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2604.05355.svg)](https://codeofpaper.com/paper/2604.05355)

Discussion