What it Thinks is Important is Important: Robustness Transfers through Input Gradients

Alvin Chan, Yi Tay, Yew-Soon Ong
12/11/2019
cs.LGcs.CVcs.NE

Official code: alvinchangw/IGAM_CVPR2020 (16 stars, last commit 2023-03-24).

Abstract

Adversarial perturbations are imperceptible changes to input pixels that can change the prediction of deep learning models. Learned weights of models robust to such perturbations are previously found to be transferable across different tasks but this applies only if the model architecture for the source and target tasks is the same. Input gradients characterize how small changes at each input pixel affect the model output. Using only natural images, we show here that training a student model's input gradients to match those of a robust teacher model can gain robustness close to a strong baseline that is robustly trained from scratch. Through experiments in MNIST, CIFAR-10, CIFAR-100 and Tiny-ImageNet, we show that our proposed method, input gradient adversarial matching, can transfer robustness across different tasks and even across different model architectures. This demonstrates that directly targeting the semantics of input gradients is a feasible way towards adversarial robustness.

View on arXivView PDF

Code Implementations(1)

alvinchangw/IGAM_CVPR2020Official100%

Implementation for What it Thinks is Important is Important: Robustness Transfers through Input Gradients (CVPR 2020 Oral)

161Shell, PythonJun 8, 20203 years ago

Similar Papers

Attribute-Guided Adversarial Training for Robustness to Natural Perturbations

87% match1 repo32020

Do Perceptually Aligned Gradients Imply Adversarial Robustness?

87% match1 repo62022

Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data

86% match2024

Non-adversarial Robustness of Deep Learning Methods for Computer Vision

85% match2023

Which Models have Perceptually-Aligned Gradients? An Explanation via Off-Manifold Robustness

85% match2023

Characterizing Model Robustness via Natural Input Gradients

85% match1 repo132024

Cite this paper

@article{chan2019what,
  title  = {What it Thinks is Important is Important: Robustness Transfers through Input Gradients},
  author = {Alvin Chan and Yi Tay and Yew-Soon Ong},
  year   = {2019},
  eprint = {1912.05699},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1912.05699}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1912.05699.svg)](https://codeofpaper.com/paper/1912.05699)

Discussion