The Test of Tests: A Framework For Differentially Private Hypothesis Testing

Zeki Kazan, Kaiyan Shi, Adam Groce, Andrew Bray
2/8/2023
stat.MEcs.CRcs.LG

No official code found. Best implementation: diff-priv-ht/test-of-tests (5 stars, last commit 2023-05-30).

Abstract

We present a generic framework for creating differentially private versions of any hypothesis test in a black-box way. We analyze the resulting tests analytically and experimentally. Most crucially, we show good practical performance for small data sets, showing that at epsilon = 1 we only need 5-6 times as much data as in the fully public setting. We compare our work to the one existing framework of this type, as well as to several individually-designed private hypothesis tests. Our framework is higher power than other generic solutions and at least competitive with (and often better than) individually-designed tests.

View on arXivView PDF

Code Implementations(1)

diff-priv-ht/test-of-testsCommunity100%
51RJan 31, 20233 years ago

Similar Papers

Better Private Distribution Testing by Leveraging Unverified Auxiliary Data

87% match2025

Differentially Private Permutation Tests: Applications to Kernel Methods

87% match2 repos12023

Differentially private Bayesian tests

85% match2024

A Statistical Viewpoint on Differential Privacy: Hypothesis Testing, Representation and Blackwell's Theorem

84% match2024

DP-SPRT: Differentially Private Sequential Probability Ratio Tests

82% match2025

Federated Nonparametric Hypothesis Testing with Differential Privacy Constraints: Optimal Rates and Adaptive Tests

81% match2024

Cite this paper

@article{kazan2023test,
  title  = {The Test of Tests: A Framework For Differentially Private Hypothesis Testing},
  author = {Zeki Kazan and Kaiyan Shi and Adam Groce and Andrew Bray},
  year   = {2023},
  eprint = {2302.04260},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2302.04260v1},
  journal = {ICML 2023 2023}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2302.04260.svg)](https://codeofpaper.com/paper/2302.04260)

Discussion