MaskCrypt: Federated Learning With Selective Homomorphic Encryption

Abstract

The federated learning paradigm protects private data from explicit leakage, yet exposing the model weights still raises serious privacy concerns with well-known attacks, such as membership inference attacks. It has been acknowledged that mechanisms such as homomorphic encryption and differential privacy can be adopted to provide a higher level of protection. However, these mechanisms may incur a formidable amount of overhead and reductions in training performance, which make them unlikely to be employed in real-world applications. In this paper, we propose <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MaskCrypt</small> , a new mechanism designed to balance the trade-off between security and practicality when homomorphic encryption is used. Rather than encrypting model updates in their entirety, <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MaskCrypt</small> applies an encryption mask to sift out a small portion of the updates for encryption. Specifically, each <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MaskCrypt</small> client adopts a gradient-guided mechanism to select the encryption mask, which aims to obfuscate the training trace by maximizing the local loss value of exposed model weights, and then sending the individual mask to a special <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Mask Consensus</i> mechanism to obtain a final mask for all clients. Our experimental results have shown convincing evidence that with a small encrypt ratio, <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MaskCrypt</small> reduced the communication overhead by up to 4.15× compared with encrypting entire model updates, yet still effectively protected the client's private data against inversion attacks, and reduced the accuracy of membership inference attacks to 49.2%.w