Improved Differentially Private Regression via Gradient Boosting

Shuai Tang, Sergul Aydore, Michael Kearns, Saeyoung Rho, Aaron Roth +3 more
3/6/2023
cs.LGcs.CR

No official code found. Best implementation: interpretml/interpret (6,885 stars, MIT, last commit 2026-04-29).

Abstract

We revisit the problem of differentially private squared error linear regression. We observe that existing state-of-the-art methods are sensitive to the choice of hyperparameters -- including the ``clipping threshold'' that cannot be set optimally in a data-independent way. We give a new algorithm for private linear regression based on gradient boosting. We show that our method consistently improves over the previous state of the art when the clipping threshold is taken to be fixed without knowledge of the data, rather than optimized in a non-private way -- and that even when we optimize the hyperparameters of competitor algorithms non-privately, our algorithm is no worse and often better. In addition to a comprehensive set of experiments, we give theoretical insights to explain this behavior.

View on arXivView PDF

Code Implementations(1)

interpretml/interpretCommunity100%
MIT

Fit interpretable models. Explain blackbox machine learning.

6,885784C, RMay 3, 20195 months agoMIT
aiartificial-intelligencebiasblackboxdifferential-privacy+14 more

Similar Papers

Near Optimal Private and Robust Linear Regression

88% match2023

Better Private Linear Regression Through Better Private Feature Selection

85% match2023

Gradient Descent with Linearly Correlated Noise: Theory and Applications to Differential Privacy

84% match2023

Differentially Private Learning Beyond the Classical Dimensionality Regime

84% match2024

Differentially Private Sliced Inverse Regression: Minimax Optimality and Algorithm

84% match2024

SoK: A Review of Differentially Private Linear Models For High-Dimensional Data

84% match2024

Cite this paper

@article{tang2023improved,
  title  = {Improved Differentially Private Regression via Gradient Boosting},
  author = {Shuai Tang and Sergul Aydore and Michael Kearns and Saeyoung Rho and Aaron Roth and Yichen Wang and Yu-Xiang Wang and Zhiwei Steven Wu},
  year   = {2023},
  eprint = {2303.03451},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2303.03451v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2303.03451.svg)](https://codeofpaper.com/paper/2303.03451)

Discussion