L-DAWA: Layer-wise Divergence Aware Weight Aggregation in Federated Self-Supervised Visual Representation Learning

Yasar Abbas Ur Rehman, Yan Gao, Pedro Gusmão, Mina Alibeigi, Jiajun Shen +1 more
2/14/2026

No official code found. Best implementation: yasar-rehman/L-DAWA (12 stars, last commit 2024-05-20).

Abstract

The ubiquity of camera-enabled devices has led to large amounts of unlabeled image data being produced at the edge. The integration of self-supervised learning (SSL) and federated learning (FL) into one coherent system can potentially offer data privacy guarantees while also advancing the quality and robustness of the learned visual representations without needing to move data around. However, client bias and divergence during FL aggregation caused by data heterogeneity limits the performance of learned visual representations on downstream tasks. In this paper, we propose a new aggregation strategy termed Layer-wise Divergence Aware Weight Aggregation (L-DAWA) to mitigate the influence of client bias and divergence during FL aggregation. The proposed method aggregates weights at the layer-level according to the measure of angular divergence between the clients’ model and the global model. Extensive experiments with cross-silo and cross-device settings on CIFAR-10/100 and Tiny ImageNet datasets demonstrate that our methods are effective and obtain new SOTA performance on both contrastive and non-contrastive SSL approaches.

View on arXivView PDF

Code Implementations(1)

yasar-rehman/L-DAWACommunity100%

This is the official repository of L-DAWA: Layer-wise Divergence Aware Weight Aggregation in Federated Self-Supervised Visual Representation Learning github

121Shell, PythonOct 30, 20232 years ago
cifar10cross-silo-federated-learningfederated-learningfederated-self-supervised-learningself-supervised-learning+1 more

Similar Papers

SelfFed: Self-Supervised Federated Learning for Data Heterogeneity and Label Scarcity in Medical Images

83% match1 repo172023

FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectors

82% match1 repo282026

FedSC: Provable Federated Self-supervised Learning with Spectral Contrastive Objective over Non-i.i.d. Data

82% match2026

OnDev-LCT: On-Device Lightweight Convolutional Transformers towards federated learning

82% match2024

FedAPA: Server-side Gradient-Based Adaptive Personalized Aggregation for Federated Learning on Heterogeneous Data

81% match2025

Task-Agnostic Federated Learning

81% match2024

Cite this paper

@article{rehman2026ldawa,
  title  = {L-DAWA: Layer-wise Divergence Aware Weight Aggregation in Federated Self-Supervised Visual Representation Learning},
  author = {Yasar Abbas Ur Rehman and Yan Gao and Pedro Gusmão and Mina Alibeigi and Jiajun Shen and Nicholas D. Lane},
  year   = {2026},
  eprint = {2307.07393},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2307.07393},
  journal = {ICCV 2023 2023}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2307.07393.svg)](https://codeofpaper.com/paper/2307.07393)

Discussion