Convergence Analysis of Sequential Split Learning on Heterogeneous Data

Yipeng Li, Xinchen Lyu
2/3/2023
cs.LG

No confident code implementation found for this paper.

Abstract

Federated Learning (FL) and Split Learning (SL) are two popular paradigms of distributed machine learning. By offloading the computation-intensive portions to the server, SL is promising for deep model training on resource-constrained devices, yet still lacking of rigorous convergence analysis. In this paper, we derive the convergence guarantees of Sequential SL (SSL, the vanilla case of SL that conducts the model training in sequence) for strongly/general/non-convex objectives on heterogeneous data. Notably, the derived guarantees suggest that SSL is better than Federated Averaging (FedAvg, the most popular algorithm in FL) on heterogeneous data. We validate the counterintuitive analysis result empirically on extremely heterogeneous data.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Convergence Analysis of Sequential Federated Learning on Heterogeneous Data

92% match1 repo82026

Convergence Analysis of Split Federated Learning on Heterogeneous Data

87% match2026

Communication and Storage Efficient Federated Split Learning

85% match2023

Non-Federated Multi-Task Split Learning for Heterogeneous Sources

84% match2024

Split Federated Learning Over Heterogeneous Edge Devices: Algorithm and Optimization

84% match2024

Faster Federated Learning with Decaying Number of Local SGD Steps

84% match2023

Cite this paper

@article{li2023convergence,
  title  = {Convergence Analysis of Sequential Split Learning on Heterogeneous Data},
  author = {Yipeng Li and Xinchen Lyu},
  year   = {2023},
  eprint = {2302.01633},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2302.01633v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2302.01633.svg)](https://codeofpaper.com/paper/2302.01633)

Discussion