SOAP: Improving and Stabilizing Shampoo using Adam for Language Modeling

Nikhil Vyas, Depen Morwani, Rosie Zhao, Itai Shapira, David Brandfonbrener +2 more
2/12/2026

No confident code implementation found for this paper.

Semantic Scholar

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Cite this paper

@article{vyas2026soap,
  title  = {SOAP: Improving and Stabilizing Shampoo using Adam for Language Modeling},
  author = {Nikhil Vyas and Depen Morwani and Rosie Zhao and Itai Shapira and David Brandfonbrener and Lucas Janson and S. Kakade},
  year   = {2026},
  url    = {https://api.semanticscholar.org/CorpusID:af0491a5a375c5b13e82fb7dcda37ad4edb4e9df},
  journal = {ICLR 2025 2025}
}

Discussion