Membership Privacy Risks of Sharpness Aware Minimization

Abstract

Optimization algorithms that seek flatter minima such as Sharpness-Aware Minimization (SAM) are widely credited with improved generalization. We ask whether such gains impact membership privacy. Surprisingly, we find that SAM is more prone to membership inference attacks than classical SGD across multiple datasets and attack methods, despite achieving lower test error. This is an intriguing phenomenon as conventional belief posits that higher membership privacy risk is associated with poor generalization. We conjecture that SAM is capable of memorizing atypical subpatterns more, leading to better generalization but higher privacy risk. We empirically validate our hypothesis by running extensive analysis on memorization and influence scores. Finally, we theoretically show how a model that captures minority subclass features more can effectively generalize better \emph{and} have higher membership privacy risk.