Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape View

Kaiyue Wen, Zhiyuan Li, Jason S. Wang, D. Hall, Percy Liang +1 more
2/12/2026
Semantic Scholar

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Cite this paper

@article{wen2026understanding,
  title  = {Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape View},
  author = {Kaiyue Wen and Zhiyuan Li and Jason S. Wang and D. Hall and Percy Liang and Tengyu Ma},
  year   = {2026},
  url    = {https://api.semanticscholar.org/CorpusID:ec5ae065f054f42ca62cb16f87c35abf1534e416},
  journal = {ICLR 2025 2025}
}

Discussion