Valid Inference for Machine Learning Model Parameters

Neil Dey, Jonathan P. Williams
2/21/2023
stat.MLcs.LGmath.ST

No official code found. Best implementation: neil-dey/valid-inference-ml-estimators (0 stars, last commit 2025-05-26).

Abstract

The parameters of a machine learning model are typically learned by minimizing a loss function on a set of training data. However, this can come with the risk of overtraining; in order for the model to generalize well, it is of great importance that we are able to find the optimal parameter for the model on the entire population -- not only on the given training sample. In this paper, we construct valid confidence sets for this optimal parameter of a machine learning model, which can be generated using only the training data without any knowledge of the population. We then show that studying the distribution of this confidence set allows us to assign a notion of confidence to arbitrary regions of the parameter space, and we demonstrate that this distribution can be well-approximated using bootstrapping techniques.

View on arXivView PDF

Code Implementations(1)

neil-dey/valid-inference-ml-estimatorsCommunity100%

Supplementary material to our paper "Valid Inference for Machine Learning Model Parameters"

00Shell, PythonFeb 21, 20231 years ago

Similar Papers

Probabilistic Consistency in Machine Learning and Its Connection to Uncertainty Quantification

81% match2025

Quantifying the Prediction Uncertainty of Machine Learning Models for Individual Data

81% match2024

Learning Against Distributional Uncertainty: On the Trade-off Between Robustness and Specificity

81% match1 repo22023

The MAPS Algorithm: Fast model-agnostic and distribution-free prediction intervals for supervised learning

80% match2024

Confidence Is All You Need for MI Attacks

80% match1 repo92023

Should We Learn Most Likely Functions or Parameters?

80% match1 repo62026

Cite this paper

@article{dey2023valid,
  title  = {Valid Inference for Machine Learning Model Parameters},
  author = {Neil Dey and Jonathan P. Williams},
  year   = {2023},
  eprint = {2302.10840},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2302.10840v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2302.10840.svg)](https://codeofpaper.com/paper/2302.10840)

Discussion