Position: Why We Must Rethink Empirical Research in Machine Learning

Moritz Herrmann, F. Julian D. Lange, Katharina Eggensperger, Giuseppe Casalicchio, Marcel Wever +5 more
5/3/2024
cs.LGstat.ML

No confident code implementation found for this paper.

Abstract

We warn against a common but incomplete understanding of empirical research in machine learning that leads to non-replicable results, makes findings unreliable, and threatens to undermine progress in the field. To overcome this alarming situation, we call for more awareness of the plurality of ways of gaining knowledge experimentally but also of some epistemic limitations. In particular, we argue most current empirical machine learning research is fashioned as confirmatory research while it should rather be considered exploratory.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Design Principles for Falsifiable, Replicable and Reproducible Empirical ML Research

86% match2024

Position: Causal Machine Learning Requires Rigorous Synthetic Experiments for Broader Adoption

86% match2025

Reproducibility in Machine Learning-based Research: Overview, Barriers and Drivers

86% match2024

Position: Machine Learning Conferences Should Establish a "Refutations and Critiques" Track

83% match2025

Reproducibility in Machine Learning-Driven Research

82% match2023

Logic-Based Explainability: Past, Present & Future

81% match2024

Cite this paper

@article{herrmann2024position,
  title  = {Position: Why We Must Rethink Empirical Research in Machine Learning},
  author = {Moritz Herrmann and F. Julian D. Lange and Katharina Eggensperger and Giuseppe Casalicchio and Marcel Wever and Matthias Feurer and David Rügamer and Eyke Hüllermeier and Anne-Laure Boulesteix and Bernd Bischl},
  year   = {2024},
  eprint = {2405.02200},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2405.02200v2},
  journal = {ICML 2024 2024}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2405.02200.svg)](https://codeofpaper.com/paper/2405.02200)

Discussion