IEEE Journal on Selected Areas in Information Theory · 2021 · 296 citations · 46 references
ComputingComputer ScienceComputer SystemsComputer SecurityDistributed SystemsMachine LearningData ProtectionFederated LearningPrivacy-preserving ProtocolsScalable Machine LearningSecret SharingDropoutModel AggregationModel-based AggregationPrivacy-preserving LearningPrivacy-preserving Model UpdatesSecure AggregationSecure Data AggregationUser-level Privacy
Federated learning is a distributed framework for training machine learning models over the data residing at mobile devices, while protecting the privacy of individual users. A major bottleneck in scaling federated learning to a large number of users is the overhead of secure model aggregation across many users. In particular, the overhead of the state-of-the-art protocols for secure model aggregation grows quadratically with the number of users. In this article, we propose the first secure aggregation framework, named Turbo-Aggregate, that in a network with N users achieves a secure aggregation overhead of O(NlogN), as opposed to O(N <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ), while tolerating up to a user dropout rate of 50%. Turbo-Aggregate employs a multi-group circular strategy for efficient model aggregation, and leverages additive secret sharing and novel coding techniques for injecting aggregation redundancy in order to handle user dropouts while guaranteeing user privacy. We experimentally demonstrate that Turbo-Aggregate achieves a total running time that grows almost linear in the number of users, and provides up to 40× speedup over the state-of-the-art protocols with up to N=200 users. Our experiments also demonstrate the impact of model size and bandwidth on the performance of Turbo-Aggregate.
46
New directions in cryptography
Whitfield Diffie, Martin E. Hellman · IEEE Transactions on Information Theory · 1976 · 14.4K citations
Adi Shamir · Communications of the ACM · 1979 · 13.5K citations · Full text
Communication-Efficient Learning of Deep Networks from Decentralized Data
H. Brendan McMahan, Eider Moore, Daniel Ramage et al. · arXiv (Cornell University) · 2016 · 5.2K citations · Full text
Federated Learning: Challenges, Methods, and Future Directions
IEEE Signal Processing Magazine · 2020 · 4.6K citations · Full text