2023 · 12 citations · 23 references
ComputingComputer ScienceMachine LearningNatural Language ProcessingPrivacy EngineeringFederated LearningFederated Learning PrivacyMachine Learning For Big DataSemi-supervised LearningSupervised LearningData DistributionLocal EnhancementModel Performance AnalysisModel Performance EvaluationPrivacyPrivacy-preserving LearningSemi-supervised Models
Federated learning (FL) is important for privacy-preserving services by training models without collecting raw user data. Most FL algorithms assume all data is annotated, which is impractical due to the high cost of labeling data in real applications. To alleviate the reliance on labeled data, semi-supervised federated learning (SSFL) has been proposed to utilize unlabeled data on clients to improve model performance. However, most existing methods either have privacy issues which share models trained on other clients, or generate pseudo-labels for unlabeled local datasets with the global model, which is usually biased towards the global data distribution. The latter may lead to sub-optimal accuracy of pseudo-labels, due to the gap between the local data distribution and the global model, especially in non-IID settings. In this paper, we propose a semi-supervised heterogeneous federated learning method with local knowledge enhancement, called FedLoKe, which aims to train an accurate global model from both labeled and unlabeled local data with non-IID distributions. Specifically, in FedLoKe, the server maintains a global model to capture global data distribution, and each client learns a local model to capture local data distribution. Since the distribution captured by the local model is aligned with the local data distribution, we utilize it to generate high-accuracy pseudo-labels of the unlabeled dataset for global model training. To prevent the local model from severely overfitting the small number of local labeled data, we further use the exponential moving average and apply the global model to generate pseudo-labels for local modeling training. Experiments on four datasets show the effectiveness of FedLoKe. Our code is available at: https://github.com/zcfinal/FedLoKe.
23
Federated Learning: Challenges, Methods, and Future Directions
IEEE Signal Processing Magazine · 2020 · 4.6K citations · Full text
FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
Kihyuk Sohn, David Berthelot, Chunliang Li et al. · arXiv (Cornell University) · 2020 · 2.3K citations · Full text
Ying Zhang, Tao Xiang, Timothy M. Hospedales et al. · 2018 · 1.8K citations · Full text
Computing, Computer Science, Artificial Intelligence In Education +9
Unsupervised Data Augmentation for Consistency Training
Qizhe Xie, Zihang Dai, Eduard Hovy et al. · arXiv (Cornell University) · 2019 · 1.6K citations · Full text