Contrastive Learning Is Spectral Clustering On Similarity Graph

Zhi-Hao Tan, Yifan Zhang, Jingqin Yang, Yang Yuan
2/12/2026

No official code found. Best implementation: yifanzhang-pro/Kernel-InfoNCE (21 stars, last commit 2024-09-17).

Abstract

Contrastive learning is a powerful self-supervised learning method, but we have a limited theoretical understanding of how it works and why it works. In this paper, we prove that contrastive learning with the standard InfoNCE loss is equivalent to spectral clustering on the similarity graph. Using this equivalence as the building block, we extend our analysis to the CLIP model and rigorously characterize how similar multi-modal objects are embedded together. Motivated by our theoretical insights, we introduce the Kernel-InfoNCE loss, incorporating mixtures of kernel functions that outperform the standard Gaussian kernel on several vision datasets. The code is available at https://github.com/yifanzhang-pro/Kernel-InfoNCE.

View on arXivView PDF

Code Implementations(1)

yifanzhang-pro/Kernel-InfoNCECommunity71%

[ICLR 2024] Contrastive Learning Is Spectral Clustering On Similarity Graph (https://arxiv.org/abs/2303.15103)

213PythonMay 23, 20232 years ago
contrastive-learningfoundation-modelspretrainingself-supervised-learning

Similar Papers

Contrastive Learning Is Spectral Clustering On Similarity Graph

100% match1 repo212023

Towards Understanding the Mechanism of Contrastive Learning via Similarity Structure: A Theoretical Analysis

87% match2023

Learning the Relation between Similarity Loss and Clustering Loss in Self-Supervised Learning

86% match1 repo502023

$\mathbb{X}$-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs

85% match2024

How does Contrastive Learning Organize Images?

84% match2023

Graph Contrastive Learning via Spectral Graph Alignment

83% match2025

Cite this paper

@article{tan2026contrastive,
  title  = {Contrastive Learning Is Spectral Clustering On Similarity Graph},
  author = {Zhi-Hao Tan and Yifan Zhang and Jingqin Yang and Yang Yuan},
  year   = {2026},
  eprint = {2303.15103},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/pdf/2303.15103},
  journal = {ICLR 2023 2023}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2303.15103.svg)](https://codeofpaper.com/paper/2303.15103)

Discussion