Electronics · 2024 · 10 citations · 17 references
ComputingComputer ScienceDistributed SystemsEdge ComputingMachine LearningEdge ArchitectureEdge Artificial IntelligenceEdge Data ProcessingEdge DevicesEdge InfrastructureFederated LearningCommunication Overhead ReductionCompression RatioEdge NodesModel CompressionPersonalized Federated LearningPruningSparse ModelsTraining Duration
Edge devices employing federated learning encounter several obstacles, including (1) the non-independent and identically distributed (Non-IID) nature of client data, (2) limitations due to communication bottlenecks, and (3) constraints on computational resources. To surmount the Non-IID data challenge, personalized federated learning has been introduced, which involves training tailored networks at the edge; nevertheless, these methods often exhibit inconsistency in performance. In response to these concerns, a novel framework for personalized federated learning that incorporates adaptive pruning of edge-side data is proposed in this paper. This approach, through a two-staged pruning process, creates customized models while ensuring strong generalization capabilities. Concurrently, by utilizing sparse models, it significantly condenses the model parameters, markedly diminishing both the computational burden and communication overhead on edge nodes. This method achieves a remarkable compression ratio of 3.7% on the Non-IID dataset FEMNIST, with the training accuracy remaining nearly unaffected. Furthermore, the total training duration is reduced by 46.4% when compared with the standard baseline method.
17
Communication-Efficient Learning of Deep Networks from Decentralized Data
H. Brendan McMahan, Eider Moore, Daniel Ramage et al. · arXiv (Cornell University) · 2016 · 5.2K citations · Full text
HRank: Filter Pruning Using High-Rank Feature Map
Mingbao Lin, Rongrong Ji, Yan Wang et al. · 2020 · 863 citations
A state-of-the-art survey on solving non-IID data in Federated Learning
Xiaodong Ma, Jia Zhu, Zhihao Lin et al. · Future Generation Computer Systems · 2022 · 362 citations