Model-Contrastive Federated Learning

Qinbin Li, Bingsheng He, Dawn Song
3/30/2021
cs.LGcs.AIcs.CV

Official code: flwrlabs/flower (7,173 stars, Apache-2.0, PyTorch, last commit 2026-04-29). 1 more implementation below.

Abstract

Federated learning enables multiple parties to collaboratively train a machine learning model without communicating their local data. A key challenge in federated learning is to handle the heterogeneity of local data distribution across parties. Although many studies have been proposed to address this challenge, we find that they fail to achieve high performance in image datasets with deep learning models. In this paper, we propose MOON: model-contrastive federated learning. MOON is a simple and effective federated learning framework. The key idea of MOON is to utilize the similarity between model representations to correct the local training of individual parties, i.e., conducting contrastive learning in model-level. Our extensive experiments show that MOON significantly outperforms the other state-of-the-art federated learning algorithms on various image classification tasks.

View on arXivView PDF

Code Implementations(2)

flwrlabs/flowerOfficial100%
PyTorchApache-2.0

Flower: A Friendly Federated AI Framework

7,1731,240C++, CSSFeb 17, 20205 months agoApache-2.0
aiandroidartificial-intelligencecppdeep-learning+15 more
Xtra-Computing/MOONCommunity100%
PyTorchMIT

Model-Contrastive Federated Learning (CVPR 2021)

29859PythonMar 28, 20214 years agoMIT
contrastive-learningcvpr2021federated-learningpytorch

Similar Papers

Relaxed Contrastive Learning for Federated Learning

89% match1 repo272024

Federated Hyperdimensional Computing

88% match2023

Federated Learning of Shareable Bases for Personalization-Friendly Image Classification

88% match2023

FedX: Unsupervised Federated Learning with Cross Knowledge Distillation

87% match1 repo732022

Multimodal Federated Learning via Contrastive Representation Ensemble

87% match1 repo1352023

Robust Asymmetric Heterogeneous Federated Learning with Corrupted Clients

86% match1 repo52025

Cite this paper

@article{li2021modelcontrastive,
  title  = {Model-Contrastive Federated Learning},
  author = {Qinbin Li and Bingsheng He and Dawn Song},
  year   = {2021},
  eprint = {2103.16257},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2103.16257}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2103.16257.svg)](https://codeofpaper.com/paper/2103.16257)

Discussion