Parameter and Memory Efficient Pretraining via Low-rank Riemannian Optimization

Zhanfeng Mo, Long-Kai Huang, Sinno Jialin Pan
2/12/2026

No official code found. Best implementation: mzf666/LORO-main (17 stars, Apache-2.0, last commit 2025-04-24).

Semantic Scholar

Code Implementations(1)

mzf666/LORO-mainCommunity62%
Apache-2.0

Official implementation of ICLR 2025 'LORO: Parameter and Memory Efficient Pretraining via Low-rank Riemannian Optimization'

170Shell, PythonJan 31, 20251 years agoApache-2.0

Similar Papers

Targeted Low-rank Refinement: Enhancing Sparse Language Models with Precision

87% match2026

SEPARATE: A Simple Low-rank Projection for Gradient Compression in Modern Large-scale Model Training Process

81% match2026

Representation Learning for Low-rank General-sum Markov Games

77% match2026

Unbiased Learning to Rank Meets Reality: Lessons from Baidu's Large-Scale Search Dataset

68% match2 repos82024

Pruning via Sparsity-indexed ODE: a Continuous Sparsity Viewpoint

68% match1 repo112026

rerankers: A Lightweight Python Library to Unify Ranking Methods

67% match1 repo1.6k2024

Cite this paper

@article{mo2026parameter,
  title  = {Parameter and Memory Efficient Pretraining via Low-rank Riemannian Optimization},
  author = {Zhanfeng Mo and Long-Kai Huang and Sinno Jialin Pan},
  year   = {2026},
  url    = {https://api.semanticscholar.org/CorpusID:44af7adad3f9cb2efdf7aef98ca400f69e415ded},
  journal = {ICLR 2025 2025}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/44af7adad3f9cb2efdf7aef98ca400f69e415ded.svg)](https://codeofpaper.com/paper/44af7adad3f9cb2efdf7aef98ca400f69e415ded)

Discussion