Incentive and Knowledge Distillation Based Federated Learning for Cross-Silo Applications

Abstract

Big data are usually characterized by heterogeneity in real-world cross-silo applications, such as healthcare, finance, and smart cities, leaving federated learning a big challenge. Further, many existing federated learning schemes fail to fully consider the diverse willingness and contributions of data providers in participation. In this paper, to address these challenges, we are motivated to propose an incentive and knowledge distillation based federated learning scheme for crosssilo applications. Specifically, we first develop a new federated learning framework, to support cooperative learning among diverse heterogeneous client models. Second, we devise an incentive mechanism, which not only stimulates workers to provide more high-quality data, but also improves clients’ enthusiasm for participating in federated learning. Third, a novel knowledge distillation algorithm is designed to deal with data heterogeneity. Extensive experiments on MNISTfflEMNIST and CIFAR10/100 datasets with both IID and Non-IID settings, demonstrate the high effectiveness of the proposed scheme, compared with stateof-the-art studies.