Efficient Wireless Federated Learning via Low-Rank Gradient Factorization

Mingzhao Guo, Dongzhu Liu, Osvaldo Simeone, Dingzhu Wen
1/15/2024
cs.ITcs.LGeess.SP

No official code found. Best implementation: MingzhaoGuo/Ota-LC (4 stars, last commit 2024-09-02).

Abstract

This paper presents a novel gradient compression method for federated learning (FL) in wireless systems. The proposed method centers on a low-rank matrix factorization strategy for local gradient compression that is based on one iteration of a distributed Jacobi successive convex approximation (SCA) at each FL round. The low-rank approximation obtained at one round is used as a "warm start" initialization for Jacobi SCA in the next FL round. A new protocol termed over-the-air low-rank compression (Ota-LC) incorporating this gradient compression method with over-the-air computation and error feedback is shown to have lower computation cost and lower communication overhead, while guaranteeing the same inference performance, as compared with existing benchmarks. As an example, when targeting a test accuracy of 70% on the Cifar-10 dataset, Ota-LC reduces total communication costs by at least 33% compared to benchmark schemes.

View on arXivView PDF

Code Implementations(1)

MingzhaoGuo/Ota-LCCommunity100%

This is the code of the paper "Efficient Wireless Federated Learning via Low-Rank Gradient Factorization"

42PythonMar 21, 20242 years ago

Similar Papers

Accelerating Wireless Federated Learning via Nesterov's Momentum and Distributed Principle Component Analysis

86% match2023

Communication-Efficient Zero-Order and First-Order Federated Learning Methods over Wireless Networks

86% match2025

Data-Aware Gradient Compression for FL in Communication-Constrained Mobile Computing

85% match2023

Rendering Wireless Environments Useful for Gradient Estimators: A Zero-Order Stochastic Federated Learning Method

85% match2024

Online-Score-Aided Federated Learning: Taming the Resource Constraints in Wireless Networks

84% match2024

A Joint Approach to Local Updating and Gradient Compression for Efficient Asynchronous Federated Learning

83% match2024

Cite this paper

@article{guo2024efficient,
  title  = {Efficient Wireless Federated Learning via Low-Rank Gradient Factorization},
  author = {Mingzhao Guo and Dongzhu Liu and Osvaldo Simeone and Dingzhu Wen},
  year   = {2024},
  eprint = {2401.07496},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2401.07496v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2401.07496.svg)](https://codeofpaper.com/paper/2401.07496)

Discussion