A Comparison of Machine Learning Methods for Data with High-Cardinality Categorical Variables

Fabio Sigrist
7/5/2023
cs.LGcs.AIstat.ML

Official code: fabsig/Compare_ML_HighCardinality_Categorical_Variables (3 stars, last commit 2024-02-29).

Abstract

High-cardinality categorical variables are variables for which the number of different levels is large relative to the sample size of a data set, or in other words, there are few data points per level. Machine learning methods can have difficulties with high-cardinality variables. In this article, we empirically compare several versions of two of the most successful machine learning methods, tree-boosting and deep neural networks, and linear mixed effects models using multiple tabular data sets with high-cardinality categorical variables. We find that, first, machine learning models with random effects have higher prediction accuracy than their classical counterparts without random effects, and, second, tree-boosting with random effects outperforms deep neural networks with random effects.

View on arXivView PDF

Code Implementations(1)

fabsig/Compare_ML_HighCardinality_Categorical_VariablesOfficial100%

Machine Learning Methods for High-Cardinality Categorical Data

30RJul 5, 20232 years ago

Similar Papers

Efficient Representations for High-Cardinality Categorical Variables in Machine Learning

84% match2025

Comparative Study on the Performance of Categorical Variable Encoders in Classification and Regression Tasks

81% match1 repo12024

The Impact of Feature Scaling In Machine Learning: Effects on Regression and Classification Tasks

80% match2025

Closing the gap on tabular data with Fourier and Implicit Categorical Features

79% match2026

Learning from higher-order statistics, efficiently: hypothesis tests, random features, and neural networks

78% match2026

(GG) MoE vs. MLP on Tabular Data

78% match2025

Cite this paper

@article{sigrist2023comparison,
  title  = {A Comparison of Machine Learning Methods for Data with High-Cardinality Categorical Variables},
  author = {Fabio Sigrist},
  year   = {2023},
  eprint = {2307.02071},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2307.02071v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2307.02071.svg)](https://codeofpaper.com/paper/2307.02071)

Discussion