Disparate Vulnerability: on the Unfairness of Privacy Attacks Against Machine Learning.

Abstract

A membership inference attack (MIA) against a machine learning model enables an attacker to determine whether a given data record was part of the model's training data or not. The effectiveness of these attacks is reported using metrics computed across the whole population (e.g., average attack accuracy). In this paper, we show that the attack success varies across different subgroups of the data (e.g., race, gender), i.e., there is \emph{disparate vulnerability}. Even if MIA's success looks no better than random guessing over the whole population, subgroups can still be vulnerable. We study the necessary and sufficient conditions for a classifier to exhibit disparate vulnerability, and we determine to what extent certain learning techniques (e.g., fairness constraints, differential privacy) can prevent it. Our work provides a theoretical framework for studying MIA attacks from a new perspective.