A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks

Dan Hendrycks, Kevin Gimpel
10/7/2016
cs.NEcs.CVcs.LG

Official code: hendrycks/error-detection (234 stars, MIT, last commit 2018-12-27).

Abstract

We consider the two related problems of detecting if an example is misclassified or out-of-distribution. We present a simple baseline that utilizes probabilities from softmax distributions. Correctly classified examples tend to have greater maximum softmax probabilities than erroneously classified and out-of-distribution examples, allowing for their detection. We assess performance by defining several tasks in computer vision, natural language processing, and automatic speech recognition, showing the effectiveness of this baseline across all. We then show the baseline can sometimes be surpassed, demonstrating the room for future research on these underexplored detection tasks.

View on arXivView PDF

Code Implementations(1)

hendrycks/error-detectionOfficial100%
MIT

A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks

23440Python, Jupyter NotebookSep 25, 20167 years agoMIT

Similar Papers

Scaling Out-of-Distribution Detection for Real-World Settings

85% match1 repo1582019

How Good Are LLMs at Out-of-Distribution Detection?

84% match1 repo142023

Augmenting Softmax Information for Selective Classification with Out-of-Distribution Data

84% match1 repo112022

A Benchmark and Evaluation for Real-World Out-of-Distribution Detection Using Vision-Language Models

84% match1 repo112025

Generalized Out-of-Distribution Detection: A Survey

84% match2 repos1.1k2021

Detecting Out-of-Distribution Examples with In-distribution Examples and Gram Matrices

83% match1 repo472019

Cite this paper

@article{hendrycks2016baseline,
  title  = {A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks},
  author = {Dan Hendrycks and Kevin Gimpel},
  year   = {2016},
  eprint = {1610.02136},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1610.02136}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1610.02136.svg)](https://codeofpaper.com/paper/1610.02136)

Discussion