ZeRO++: Extremely Efficient Collective Communication for Large Model Training

Guanhua Wang, Heyang Qin, Sam Ade Jacobs, Xiaoxia Wu, Connor Holmes +6 more
2/12/2026

No confident code implementation found for this paper.

Semantic Scholar

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Cite this paper

@article{wang2026zero,
  title  = {ZeRO++: Extremely Efficient Collective Communication for Large Model Training},
  author = {Guanhua Wang and Heyang Qin and Sam Ade Jacobs and Xiaoxia Wu and Connor Holmes and Zhewei Yao and Samyam Rajbhandari and Olatunji Ruwase and Feng Yan and Lei Yang and Yuxiong He},
  year   = {2026},
  url    = {https://api.semanticscholar.org/CorpusID:8dd04dd93055d1f084bebb13795852c72f0e74ce},
  journal = {ICLR 2024 2024}
}

Discussion