Flaws of ImageNet, Computer Vision's Favourite Dataset

Nikita Kisel, Illia Volkov, Katerina Hanzelkova, Klara Janouskova, Jiri Matas
11/26/2024
cs.CV

No confident code implementation found for this paper. 3 possible matches can be shown below.

Abstract

Since its release, ImageNet-1k dataset has become a gold standard for evaluating model performance. It has served as the foundation for numerous other datasets and training tasks in computer vision. As models have improved in accuracy, issues related to label correctness have become increasingly apparent. In this blog post, we analyze the issues in the ImageNet-1k dataset, including incorrect labels, overlapping or ambiguous class definitions, training-evaluation domain shifts, and image duplicates. The solutions for some problems are straightforward. For others, we hope to start a broader conversation about refining this influential dataset to better serve future research.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 3 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Leveraging Human-Machine Interactions for Computer Vision Dataset Quality Enhancement

86% match1 repo2024

Vision Large Language Models Are Good Noise Handlers in Engagement Analysis

82% match2025

On Evaluation of Vision Datasets and Models using Human Competency Frameworks

82% match2024

The Impact of the Single-Label Assumption in Image Recognition Benchmarking

82% match2024

Demystifying CLIP Data

82% match1 repo1.9k2026

JAMMEval: A Refined Collection of Japanese Benchmarks for Reliable VLM Evaluation

82% match2026

Cite this paper

@article{kisel2024flaws,
  title  = {Flaws of ImageNet, Computer Vision's Favourite Dataset},
  author = {Nikita Kisel and Illia Volkov and Katerina Hanzelkova and Klara Janouskova and Jiri Matas},
  year   = {2024},
  eprint = {2412.00076},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2412.00076v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2412.00076.svg)](https://codeofpaper.com/paper/2412.00076)

Discussion