A Rigorous Behavior Assessment of CNNs Using a Data-Domain Sampling Regime

Shuning Jiang, Wei-Lun Chao, Daniel Haehn, Hanspeter Pfister, Jian Chen
7/5/2025
cs.LGcs.CVcs.HC

No confident code implementation found for this paper.

Abstract

We present a data-domain sampling regime for quantifying CNNs' graphic perception behaviors. This regime lets us evaluate CNNs' ratio estimation ability in bar charts from three perspectives: sensitivity to training-test distribution discrepancies, stability to limited samples, and relative expertise to human observers. After analyzing 16 million trials from 800 CNNs models and 6,825 trials from 113 human participants, we arrived at a simple and actionable conclusion: CNNs can outperform humans and their biases simply depend on the training-test distance. We show evidence of this simple, elegant behavior of the machines when they interpret visualization images. osf.io/gfqc3 provides registration, the code for our sampling regime, and experimental results.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Generalization of CNNs on Relational Reasoning with Bar Charts

83% match1 repo12025

ImageNet-trained CNNs are not biased towards texture: Revisiting feature reliance through controlled suppression

82% match1 repo232025

Linking convolutional kernel size to generalization bias in face analysis CNNs

82% match2023

Which Backbone to Use: A Resource-efficient Domain Specific Comparison for Computer Vision

81% match1 repo122024

Using Human-like Mechanism to Weaken Effect of Pre-training Weight Bias in Face-Recognition Convolutional Neural Network

81% match2023

CNNtention: Can CNNs do better with Attention?

81% match1 repo62024

Cite this paper

@article{jiang2025rigorous,
  title  = {A Rigorous Behavior Assessment of CNNs Using a Data-Domain Sampling Regime},
  author = {Shuning Jiang and Wei-Lun Chao and Daniel Haehn and Hanspeter Pfister and Jian Chen},
  year   = {2025},
  eprint = {2507.03866},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2507.03866v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2507.03866.svg)](https://codeofpaper.com/paper/2507.03866)

Discussion