FedVF: Personalized Federated Learning Based on Layer-wise Parameter Updates with Variable Frequency

Abstract

Federated learning is a new and increasingly popular distributed machine learning paradigm, which can employ multiple clients such as mobile phones to collaboratively train a deep neural network model under the coordination of the central server. The raw data is always kept locally on the clients, and only the model parameters are communicated between clients and the server, so that data privacy can be largely preserved. To cope with the effect of not independent and identically distributed (Non-IID) data, personalized federated learning that can provide personalized and customized models for clients participating in federated training has been proposed and widely studied. In this paper, we propose a personalized federated learning algorithm that can provide a personalized local model for each client while storing the latest global model on the central server through only one federated training. The deep neural network model is divided into two parts of global layers and personalized layers, and the whole training process is partitioned into earlier and later stages. Based on the division of layer-wise parameters, the cumulative learning strategy of updating parameters on different layers with variable frequency is adopted to better personalize local models in the later stage based on the global features learned in the earlier stage. Compared with existing personalized federated learning algorithms, our proposed algorithm can achieve a good balance between the personalized model and the global model in federated learning, and performs better in communication efficiency and model accuracy.