On the Non-decoupling of Supervised Fine-tuning and Reinforcement Learning in Post-training

Xueyan Niu, Bo Bai, Wei Han, Weixi Zhang
1/12/2026
cs.LGcs.AIcs.IT

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

Post-training of large language models routinely interleaves supervised fine-tuning (SFT) with reinforcement learning (RL). These two methods have different objectives: SFT minimizes the cross-entropy loss between model outputs and expert responses, while RL maximizes reward signals derived from human preferences or rule-based verifiers. Modern reasoning models have widely adopted the practice of alternating SFT and RL training. However, there is no theoretical account of whether they can be decoupled. We prove that decoupling is impossible in either order: (1) SFT-then-RL coupling: RL increases SFT loss under SFT optimality and (2) RL-then-SFT coupling: SFT lowers the reward achieved by RL. Experiments on Qwen3-0.6B confirm the predicted degradation, verifying that SFT and RL cannot be separated without loss of prior performance in the post-training

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Mitigating Forgetting Between Supervised and Reinforcement Learning Yields Stronger Reasoners

88% match2025

UFT: Unifying Supervised and Reinforcement Fine-Tuning

88% match1 repo312025

Anchored Supervised Fine-Tuning

87% match1 repo502025

SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoning

87% match2025

Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Training

87% match2025

Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Training

87% match2025

Cite this paper

@article{niu2026nondecoupling,
  title  = {On the Non-decoupling of Supervised Fine-tuning and Reinforcement Learning in Post-training},
  author = {Xueyan Niu and Bo Bai and Wei Han and Weixi Zhang},
  year   = {2026},
  eprint = {2601.07389},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2601.07389v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2601.07389.svg)](https://codeofpaper.com/paper/2601.07389)

Discussion