Rethinking Robust Contrastive Learning from the Adversarial Perspective

Fatemeh Ghofrani, Mehdi Yaghouti, Pooyan Jamshidi
2/5/2023
cs.LGcs.CV

Official code: softsys4ai/CL-Robustness (3 stars, last commit 2024-02-26).

Abstract

To advance the understanding of robust deep learning, we delve into the effects of adversarial training on self-supervised and supervised contrastive learning alongside supervised learning. Our analysis uncovers significant disparities between adversarial and clean representations in standard-trained networks across various learning algorithms. Remarkably, adversarial training mitigates these disparities and fosters the convergence of representations toward a universal set, regardless of the learning scheme used. Additionally, increasing the similarity between adversarial and clean representations, particularly near the end of the network, enhances network robustness. These findings offer valuable insights for designing and training effective and robust deep learning networks. Our code is released at \textcolor{magenta}{\url{https://github.com/softsys4ai/CL-Robustness}}.

View on arXivView PDF

Code Implementations(1)

softsys4ai/CL-RobustnessOfficial100%

A comprehensive study to compare the robustness of contrastive and supervised contrastive with standard supervised learning under different adversarial training scenarios

31Python, Jupyter NotebookFeb 2, 20232 years ago

Similar Papers

Generalization Bounds for Robust Contrastive Learning: From Theory to Practice

89% match2023

Better Representations via Adversarial Training in Pre-Training: A Theoretical Perspective

89% match2024

C-LEAD: Contrastive Learning for Enhanced Adversarial Defense

87% match2025

Generalization Bounds for Adversarial Contrastive Learning

87% match2023

Splitting the Difference on Adversarial Training

86% match2023

Combining Adversaries with Anti-adversaries in Training

86% match1 repo72023

Cite this paper

@article{ghofrani2023rethinking,
  title  = {Rethinking Robust Contrastive Learning from the Adversarial Perspective},
  author = {Fatemeh Ghofrani and Mehdi Yaghouti and Pooyan Jamshidi},
  year   = {2023},
  eprint = {2302.02502},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2302.02502v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2302.02502.svg)](https://codeofpaper.com/paper/2302.02502)

Discussion