Subsampled R\\'enyi Differential Privacy and Analytical Moments\n Accountant

Abstract

We study the problem of subsampling in differential privacy (DP), a question\nthat is the centerpiece behind many successful differentially private machine\nlearning algorithms. Specifically, we provide a tight upper bound on the\nR\\'enyi Differential Privacy (RDP) (Mironov, 2017) parameters for algorithms\nthat: (1) subsample the dataset, and then (2) applies a randomized mechanism M\nto the subsample, in terms of the RDP parameters of M and the subsampling\nprobability parameter. Our results generalize the moments accounting technique,\ndeveloped by Abadi et al. (2016) for the Gaussian mechanism, to any subsampled\nRDP mechanism.\n