SignSGD: Fault-Tolerance to Blind and Byzantine Adversaries

Jason Akoun, Sebastien Meyer
2/4/2022
cs.LGstat.ML

Official code: jasonakoun/signsgd-fault-tolerance (3 stars, last commit 2022-01-24).

Abstract

Distributed learning has become a necessity for training ever-growing models by sharing calculation among several devices. However, some of the devices can be faulty, deliberately or not, preventing the proper convergence. As a matter of fact, the baseline distributed SGD algorithm does not converge in the presence of one Byzantine adversary. In this article we focus on the more robust SignSGD algorithm derived from SGD. We provide an upper bound for the convergence rate of SignSGD proving that this new version is robust to Byzantine adversaries. We implemented SignSGD along with Byzantine strategies attempting to crush the learning process. Therefore, we provide empirical observations from our experiments to support our theory. Our code is available on GitHub https://github.com/jasonakoun/signsgd-fault-tolerance and our experiments are reproducible by using the provided parameters.

View on arXivView PDF

Code Implementations(1)

jasonakoun/signsgd-fault-toleranceOfficial100%

Here is a basic implementation of SignSGD to assess its robustness to adversaries.

30PythonJan 20, 20224 years ago

Similar Papers

SignSGD with Federated Defense: Harnessing Adversarial Attacks through Gradient Sign Decoding

84% match2024

Game of Trojans: A Submodular Byzantine Approach

84% match2022

Coded Robust Aggregation for Distributed Learning under Byzantine Attacks

83% match2025

Testing Robustness Against Unforeseen Adversaries

83% match2 repos802019

Byzantine-Robust Learning on Heterogeneous Data via Gradient Splitting

83% match1 repo152023

Towards Robust Learning to Optimize with Theoretical Guarantees

82% match1 repo42025

Cite this paper

@article{akoun2022signsgd,
  title  = {SignSGD: Fault-Tolerance to Blind and Byzantine Adversaries},
  author = {Jason Akoun and Sebastien Meyer},
  year   = {2022},
  eprint = {2202.02085},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2202.02085}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2202.02085.svg)](https://codeofpaper.com/paper/2202.02085)

Discussion