Why are Saliency Maps Noisy? Cause of and Solution to Noisy Saliency Maps

Beomsu Kim, Junghoon Seo, SeungHyun Jeon, Jamyoung Koo, Jeongyeol Choe +1 more
2/13/2019
cs.LGcs.CVstat.ML

Official code: 1202kbs/Rectified-Gradient (34 stars, last commit 2019-09-13).

Abstract

Saliency Map, the gradient of the score function with respect to the input, is the most basic technique for interpreting deep neural network decisions. However, saliency maps are often visually noisy. Although several hypotheses were proposed to account for this phenomenon, there are few works that provide rigorous analyses of noisy saliency maps. In this paper, we firstly propose a new hypothesis that noise may occur in saliency maps when irrelevant features pass through ReLU activation functions. Then, we propose Rectified Gradient, a method that alleviates this problem through layer-wise thresholding during backpropagation. Experiments with neural networks trained on CIFAR-10 and ImageNet showed effectiveness of our method and its superiority to other attribution methods.

View on arXivView PDF

Code Implementations(1)

1202kbs/Rectified-GradientOfficial100%

Official repository for "Why are Saliency Maps Noisy? Cause of and Solution to Noisy Saliency Maps".

348Python, Jupyter NotebookJan 24, 20197 years ago

Similar Papers

Rethinking gradient weights' influence over saliency map estimation

87% match2022

XRAI: Better Attributions Through Regions

87% match1 repo9962019

Verifying Attention Robustness of Deep Neural Networks against Semantic Perturbations

86% match2022

Smooth Deep Saliency

86% match1 repo2024

Forward Learning for Gradient-based Black-box Saliency Map Generation

86% match2024

Full-Gradient Representation for Neural Network Visualization

85% match1 repo2102019

Cite this paper

@article{kim2019saliency,
  title  = {Why are Saliency Maps Noisy? Cause of and Solution to Noisy Saliency Maps},
  author = {Beomsu Kim and Junghoon Seo and SeungHyun Jeon and Jamyoung Koo and Jeongyeol Choe and Taegyun Jeon},
  year   = {2019},
  eprint = {1902.04893},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1902.04893}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1902.04893.svg)](https://codeofpaper.com/paper/1902.04893)

Discussion