Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models

Pengyi Li, Matvey Skripkin, Alexander Zubrey, Andrey Kuznetsov, Ivan Oseledets
6/5/2025
cs.CLcs.LG

No confident code implementation found for this paper. 10 possible matches can be shown below.

Abstract

Large language models (LLMs) excel at reasoning, yet post-training remains critical for aligning their behavior with task goals. Existing reinforcement learning (RL) methods often depend on costly human annotations or external reward models. We propose Reinforcement Learning via Self-Confidence (RLSC), which uses the model's own confidence as reward signals-eliminating the need for labels, preference models, or reward engineering. Applied to Qwen2.5-Math-7B with only 16 samples per question and 10 or 20 training steps, RLSC improves accuracy by +13.4% on AIME2024, +21.2% on MATH500, +21.7% on Minerva Math, +20.8% on Olympiadbench, and +9.7% on AMC23. RLSC provides a simple, scalable post-training method for inference models, requiring only a small number of samples and unlabelled supervision.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 10 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Post-Training Large Language Models via Reinforcement Learning from Self-Feedback

93% match2025

Rewarding Doubt: A Reinforcement Learning Approach to Calibrated Confidence Expression of Large Language Models

91% match2025

ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs

91% match2025

SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales

91% match1 repo1132024

Beyond Correctness: Confidence-Aware Reward Modeling for Enhancing Large Language Model Reasoning

90% match1 repo32025

Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty

90% match1 repo882025

Cite this paper

@article{li2025confidence,
  title  = {Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models},
  author = {Pengyi Li and Matvey Skripkin and Alexander Zubrey and Andrey Kuznetsov and Ivan Oseledets},
  year   = {2025},
  eprint = {2506.06395},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2506.06395v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2506.06395.svg)](https://codeofpaper.com/paper/2506.06395)

Discussion