Robust Asynchronous Federated Learning With Time-Weighted and Stale Model Aggregation

Abstract

Federated Learning (FL) ensures collaborative learning among multiple clients while maintaining data locally. However, the traditional synchronous FL solutions have lower accuracy and require more communication time in scenarios where most devices drop out during learning. Therefore, we propose an <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Asy</u> nchronous <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</u> ederated <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</u> earning (AsyFL) scheme using time-weighted and stale model aggregation, which effectively solves the problem of poor model performance due to the heterogeneity of devices. Then, we integrate Symmetric Homomorphic Encryption (SHE) into AsyFL to propose <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Asy</u> nchronous <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">P</u> rivacy- <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">P</u> reserving <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</u> ederated <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</u> earning (Asy-PPFL), which protects the privacy of clients and achieves lightweight computing. Privacy analysis shows that Asy-PPFL is indistinguishable under Known Plaintext Attack (KPA) and convergence analysis proves the effectiveness of our schemes. A large number of experiments show that AsyFL and Asy-PPFL can achieve the highest accuracy of 58.40% and 58.26% on Cifar-10 dataset when most clients (i.e., 80%) are offline or delayed, respectively.