IEEE INFOCOM 2022 - IEEE Conference on Computer Communications · 2022 · 58 citations · 27 references
ComputingCommunicationComputer ScienceAlgorithmsMachine LearningOptimizationData CompressionDeep LearningDistributed AlgorithmsFederated LearningLossy Data CompressionOptimization For Machine LearningCommunication Overhead ReductionCompressionCompression AlgorithmsConvergence Rate AnalysisFederated Training ArchitectureLearningModel Accuracy AssessmentPerformance Overhead
Federated Learning (FL) incurs high communication overhead, which can be greatly alleviated by compression for model updates. Yet the tradeoff between compression and model accuracy in the networked environment remains unclear and, for simplicity, most implementations adopt a fixed compression rate only. In this paper, we for the first time systematically examine this tradeoff, identifying the influence of the compression error on the final model accuracy with respect to the learning rate. Specifically, we factor the compression error of each global iteration into the convergence rate analysis under both strongly convex and non-convex loss functions. We then present an adaptation framework to maximize the final model accuracy by strategically adjusting the compression rate in each iteration. We have discussed the key implementation issues of our framework in practical networks with representative compression algorithms. Experiments over the popular MNIST and CIFAR-10 datasets confirm that our solution effectively reduces network traffic yet maintains high model accuracy in FL.
27
Edge Computing: Vision and Challenges
Weisong Shi, Jie Cao, Quan Zhang et al. · IEEE Internet of Things Journal · 2016 · 7.9K citations
Qiang Yang, Yang Liu, Tianjian Chen et al. · ACM Transactions on Intelligent Systems and Technology · 2019 · 5.9K citations
Federated Learning: Strategies for Improving Communication Efficiency
Jakub Konečný, H. Brendan McMahan, Felix X. Yu et al. · arXiv (Cornell University) · 2016 · 3K citations · Full text