Byzantine-Robust Learning on Heterogeneous Data via Gradient Splitting

Yuchen Liu, Chen Chen, Lingjuan Lyu, Fangzhao Wu, Sai Wu +1 more
2/1/2023

No official code found. Best implementation: YuchenLiu-a/byzantine-gas (15 stars, last commit 2023-06-09).

Abstract

Federated learning has exhibited vulnerabilities to Byzantine attacks, where the Byzantine attackers can send arbitrary gradients to a central server to destroy the convergence and performance of the global model. A wealth of robust AGgregation Rules (AGRs) have been proposed to defend against Byzantine attacks. However, Byzantine clients can still circumvent robust AGRs when data is non-Identically and Independently Distributed (non-IID). In this paper, we first reveal the root causes of performance degradation of current robust AGRs in non-IID settings: the curse of dimensionality and gradient heterogeneity. In order to address this issue, we propose GAS, a \shorten approach that can successfully adapt existing robust AGRs to non-IID settings. We also provide a detailed convergence analysis when the existing robust AGRs are combined with GAS. Experiments on various real-world datasets verify the efficacy of our proposed GAS. The implementation code is provided in https://github.com/YuchenLiu-a/byzantine-gas.

View on arXivView PDF

Code Implementations(1)

YuchenLiu-a/byzantine-gasCommunity100%

Official Implementation of ICML'23 "Byzantine-Robust Learning on Heterogeneous Data via Gradient Splitting".

152PythonMay 29, 20233 years ago

Similar Papers

Do Perceptually Aligned Gradients Imply Adversarial Robustness?

85% match1 repo62022

Testing Robustness Against Unforeseen Adversaries

84% match2 repos802019

Coded Robust Aggregation for Distributed Learning under Byzantine Attacks

83% match2025

SignSGD: Fault-Tolerance to Blind and Byzantine Adversaries

83% match1 repo32022

Efficient Loss Function by Minimizing the Detrimental Effect of Floating-Point Errors on Gradient-Based Attacks

82% match2023

What Makes VLMs Robust? Towards Reconciling Robustness and Accuracy in Vision-Language Models

82% match2026

Cite this paper

@article{liu2023byzantinerobust,
  title  = {Byzantine-Robust Learning on Heterogeneous Data via Gradient Splitting},
  author = {Yuchen Liu and Chen Chen and Lingjuan Lyu and Fangzhao Wu and Sai Wu and Gang Chen},
  year   = {2023},
  eprint = {2302.06079},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2302.06079},
  journal = {ICML 2023 2023}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2302.06079.svg)](https://codeofpaper.com/paper/2302.06079)

Discussion