CatBoost: unbiased boosting with categorical features

Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, Andrey Gulin
6/28/2017
cs.LG

Official code: catboost/catboost (9,133 stars, Apache-2.0, last commit 2026-04-29).

Abstract

This paper presents the key algorithmic techniques behind CatBoost, a new gradient boosting toolkit. Their combination leads to CatBoost outperforming other publicly available boosting implementations in terms of quality on a variety of datasets. Two critical algorithmic advances introduced in CatBoost are the implementation of ordered boosting, a permutation-driven alternative to the classic algorithm, and an innovative algorithm for processing categorical features. Both techniques were created to fight a prediction shift caused by a special kind of target leakage present in all currently existing implementations of gradient boosting algorithms. In this paper, we provide a detailed analysis of this problem and demonstrate that proposed algorithms solve it effectively, leading to excellent empirical results.

View on arXivView PDF

Code Implementations(1)

catboost/catboostOfficial100%
Apache-2.0

A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

9,1331,345C, RJul 18, 20175 months agoApache-2.0
big-datacatboostcategorical-featurescoremlcuda+13 more

Similar Papers

CatBoost: gradient boosting with categorical features support

92% match1 repo9.1k2018

Gradient and Newton Boosting for Classification and Regression

85% match2 repos612018

Benchmarking state-of-the-art gradient boosting algorithms for classification

84% match2023

RashomonGB: Analyzing the Rashomon Effect and Mitigating Predictive Multiplicity in Gradient Boosting

82% match2024

From Point to probabilistic gradient boosting for claim frequency and severity prediction

81% match2024

Revisiting Agnostic Boosting

81% match2025

Cite this paper

@article{prokhorenkova2017catboost,
  title  = {CatBoost: unbiased boosting with categorical features},
  author = {Liudmila Prokhorenkova and Gleb Gusev and Aleksandr Vorobev and Anna Veronika Dorogush and Andrey Gulin},
  year   = {2017},
  eprint = {1706.09516},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1706.09516}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1706.09516.svg)](https://codeofpaper.com/paper/1706.09516)

Discussion