Adaptive conformal classification with noisy labels

Matteo Sesia, Y. X. Rachel Wang, Xin Tong
9/10/2023
stat.MEcs.LGmath.ST

Official code: msesia/conformal-label-noise (5 stars, MIT, last commit 2024-06-19).

Abstract

This paper develops novel conformal prediction methods for classification tasks that can automatically adapt to random label contamination in the calibration sample, leading to more informative prediction sets with stronger coverage guarantees compared to state-of-the-art approaches. This is made possible by a precise characterization of the effective coverage inflation (or deflation) suffered by standard conformal inferences in the presence of label contamination, which is then made actionable through new calibration algorithms. Our solution is flexible and can leverage different modeling assumptions about the label contamination process, while requiring no knowledge of the underlying data distribution or of the inner workings of the machine-learning classifier. The advantages of the proposed methods are demonstrated through extensive simulations and an application to object classification with the CIFAR-10H image data set.

View on arXivView PDF

Code Implementations(1)

msesia/conformal-label-noiseOfficial
MIT

This repository implements conformal prediction methods for classification tasks that can automatically adapt to random label contamination in the calibration sample.

51R, ShellSep 10, 20232 years agoMIT
classificationcomputer-visionconformal-predictionlabel-noiseuncertainty-quantification

Similar Papers

Noise-Adaptive Conformal Classification with Marginal Coverage

91% match1 repo12025

Conformal Prediction of Classifiers with Many Classes based on Noisy Labels

89% match1 repo32025

A Conformal Prediction Score that is Robust to Label Noise

86% match2024

Sparse Activations as Conformal Predictors

85% match1 repo22025

Split Conformal Prediction under Data Contamination

85% match2024

Conformal Prediction for Class-wise Coverage via Augmented Label Rank Calibration

85% match1 repo62024

Cite this paper

@article{sesia2023adaptive,
  title  = {Adaptive conformal classification with noisy labels},
  author = {Matteo Sesia and Y. X. Rachel Wang and Xin Tong},
  year   = {2023},
  eprint = {2309.05092},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2309.05092v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2309.05092.svg)](https://codeofpaper.com/paper/2309.05092)

Discussion