TabRepo: A Large Scale Repository of Tabular Model Evaluations and its AutoML Applications

David Salinas, Nick Erickson
11/6/2023
cs.LGcs.AI

Official code: autogluon/autogluon (10,763 stars, Apache-2.0, PyTorch, last commit 2026-04-28). 1 more implementation below.

Abstract

We introduce TabRepo, a new dataset of tabular model evaluations and predictions. TabRepo contains the predictions and metrics of 1310 models evaluated on 200 classification and regression datasets. We illustrate the benefit of our dataset in multiple ways. First, we show that it allows to perform analysis such as comparing Hyperparameter Optimization against current AutoML systems while also considering ensembling at marginal cost by using precomputed model predictions. Second, we show that our dataset can be readily leveraged to perform transfer-learning. In particular, we show that applying standard transfer-learning techniques allows to outperform current state-of-the-art tabular systems in accuracy, runtime and latency.

View on arXivView PDF

Code Implementations(2)

autogluon/tabarenaOfficial100%
Apache-2.0

A Living Benchmark for Machine Learning on Tabular Data

33378C++, ShellMay 15, 20235 months agoApache-2.0
autogluon/autogluonOfficial68%
PyTorchApache-2.0

Fast and Accurate ML in 3 Lines of Code

10,7631,195Shell, PythonJul 29, 20195 months agoApache-2.0
autogluonautomated-machine-learningautomlcomputer-visiondata-science+15 more

Similar Papers

Large Scale Transfer Learning for Tabular Data via Language Modeling

85% match2 repos712024

A Data-Centric Perspective on Evaluating Machine Learning Models for Tabular Data

85% match1 repo162024

TabReD: Analyzing Pitfalls and Filling the Gaps in Tabular Deep Learning Benchmarks

83% match1 repo952024

TabReD: Analyzing Pitfalls and Filling the Gaps in Tabular Deep Learning Benchmarks

83% match2 repos1.1k2024

TabArena: A Living Benchmark for Machine Learning on Tabular Data

83% match1 repo3332025

PMLBmini: A Tabular Classification Benchmark Suite for Data-Scarce Applications

83% match1 repo112024

Cite this paper

@article{salinas2023tabrepo,
  title  = {TabRepo: A Large Scale Repository of Tabular Model Evaluations and its AutoML Applications},
  author = {David Salinas and Nick Erickson},
  year   = {2023},
  eprint = {2311.02971},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2311.02971v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2311.02971.svg)](https://codeofpaper.com/paper/2311.02971)

Discussion