An Efficient Subspace Algorithm for Federated Learning on Heterogeneous Data

Jiaojiao Zhang, Yuqi Xu, Kun Yuan
9/5/2025
cs.LGcs.DC

No official code found. Best implementation: JimmyAwoe/FedSub (2 stars, MIT, last commit 2025-12-11).

Abstract

This work addresses the key challenges of applying federated learning to large-scale deep neural networks, particularly the issue of client drift due to data heterogeneity across clients and the high costs of communication, computation, and memory. We propose FedSub, an efficient subspace algorithm for federated learning on heterogeneous data. Specifically, FedSub utilizes subspace projection to guarantee local updates of each client within low-dimensional subspaces, thereby reducing communication, computation, and memory costs. Additionally, it incorporates low-dimensional dual variables to mitigate client drift. We provide convergence analysis that reveals the impact of key factors such as step size and subspace projection matrices on convergence. Experimental results demonstrate its efficiency.

View on arXivView PDF

Code Implementations(1)

JimmyAwoe/FedSubCommunity59%
MIT

Official implementation of FedSub: An efficient subspace algorithm for federated learning on heterogeneous data.

20Shell, PythonMar 29, 202510 months agoMIT

Similar Papers

Gradient Projection onto Historical Descent Directions for Communication-Efficient Federated Learning

85% match2025

FedLALR: Client-Specific Adaptive Learning Rates Achieve Linear Speedup for Non-IID Data

85% match2023

On the Effectiveness of Partial Variance Reduction in Federated Learning with Heterogeneous Data

85% match2026

Federated Learning over Connected Modes

85% match1 repo52024

Federated Learning over Connected Modes

85% match1 repo52026

Adaptive Parameterization of Deep Learning Models for Federated Learning

84% match2023

Cite this paper

@article{zhang2025efficient,
  title  = {An Efficient Subspace Algorithm for Federated Learning on Heterogeneous Data},
  author = {Jiaojiao Zhang and Yuqi Xu and Kun Yuan},
  year   = {2025},
  eprint = {2509.05213},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2509.05213v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2509.05213.svg)](https://codeofpaper.com/paper/2509.05213)

Discussion