Group and Attack: Auditing Differential Privacy

Abstract

(ε, δ) differential privacy has seen increased adoption recently, especially in private machine learning applications. While this privacy definition allows provably limiting the amount of information leaked by an algorithm, practical implementations of differentially private algorithms often contain subtle vulnerabilities. This motivates the need for effective tools that can audit (ε, δ) differential privacy algorithms before deploying them in the real world. However, existing state-of-the-art-tools for auditing (ε, δ) differential privacy directly extend the tools for ε-differential privacy by fixing either ε or δ in the violation search, inherently restricting their ability to efficiently discover violations of (ε, δ) differential privacy.