Data Heterogeneity and Forgotten Labels in Split Federated Learning

Joana Tirana, Dimitra Tsigkari, David Solans Noguero, Nicolas Kourtellis
11/12/2025
cs.LG

No official code found. Best implementation: jtirana98/Hydra-CF-in-SFL (9 stars, MIT, last commit 2025-11-15).

Abstract

In Split Federated Learning (SFL), the clients collaboratively train a model with the help of a server by splitting the model into two parts. Part-1 is trained locally at each client and aggregated by the aggregator at the end of each round. Part-2 is trained at a server that sequentially processes the intermediate activations received from each client. We study the phenomenon of catastrophic forgetting (CF) in SFL in the presence of data heterogeneity. In detail, due to the nature of SFL, local updates of part-1 may drift away from global optima, while part-2 is sensitive to the processing sequence, similar to forgetting in continual learning (CL). Specifically, we observe that the trained model performs better in classes (labels) seen at the end of the sequence. We investigate this phenomenon with emphasis on key aspects of SFL, such as the processing order at the server and the cut layer. Based on our findings, we propose Hydra, a novel mitigation method inspired by multi-head neural networks and adapted for the SFL's setting. Extensive numerical evaluations show that Hydra outperforms baselines and methods from the literature.

View on arXivView PDF

Code Implementations(1)

jtirana98/Hydra-CF-in-SFLCommunity56%
MIT

/* paper: Data Heterogeneity and Forgotten Labels in Split Federated Learning (AAAI 26) */ Framework for evaluating Catastrophic Forgetting (CF) in SFL, SL, and FL. Also, contains Hydra an extension for SplitFedV2 to tackle CF.

90Shell, PythonNov 10, 202511 months agoMIT

Similar Papers

Re-Weighted Softmax Cross-Entropy to Control Forgetting in Federated Learning

86% match2023

Preservation of the Global Knowledge by Not-True Distillation in Federated Learning

86% match1 repo942021

KoReA-SFL: Knowledge Replay-based Split Federated Learning Against Catastrophic Forgetting

86% match2024

Towards Fast and Stable Federated Learning: Confronting Heterogeneity via Knowledge Anchor

85% match1 repo2023

FedDUAL: A Dual-Strategy with Adaptive Loss and Dynamic Aggregation for Mitigating Data Heterogeneity in Federated Learning

85% match1 repo12024

Handling Spatial-Temporal Data Heterogeneity for Federated Continual Learning via Tail Anchor

84% match2024

Cite this paper

@article{tirana2025data,
  title  = {Data Heterogeneity and Forgotten Labels in Split Federated Learning},
  author = {Joana Tirana and Dimitra Tsigkari and David Solans Noguero and Nicolas Kourtellis},
  year   = {2025},
  eprint = {2511.09736},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2511.09736v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2511.09736.svg)](https://codeofpaper.com/paper/2511.09736)

Discussion