Fair Scratch Tickets: Finding Fair Sparse Networks without Weight Training

Abstract

Recent studies suggest that computer vision models come at the risk of compromising fairness. There are exten-sive works to alleviate unfairness in computer vision using pre-processing, in-processing, and post-processing meth-ods. In this paper, we lead a novel fairness-aware learning paradigm for in-processing methods through the lens of the lottery ticket hypothesis (LTH) in the context of computer vision fairness. We randomly initialize a dense neural net-work and find appropriate binary masks for the weights to obtain fair sparse subnetworks without any weight training. Interestingly, to the best of our knowledge, we are the first to discover that such sparse subnetworks with inborn fair-ness exist in randomly initialized networks, achieving an accuracy-fairness trade-off comparable to that of dense neural networks trained with existing fairness-aware in-processing approaches. We term these fair subnetworks as Fair Scratch Tickets (FSTs). We also theoretically pro-vide fairness and accuracy guarantees for them. In our experiments, we investigate the existence of FSTs on var-ious datasets, target attributes, random initialization meth-ods, sparsity patterns, and fairness surrogates. We also find that FSTs can transfer across datasets and investigate other properties of FSTs.