Federated Learning Using Sparse-Adaptive Model Selection for Embedded Edge Computing

Abstract

Federated learning (FL) is a distributed machine learning paradigm in which multiple clients collaboratively learn a global model without sharing local training data. This study presents an adaptive model selection algorithm to enhance existing aggregation for FL. The proposed method uses sparsely learned local models and adaptively selects the best models based on the entropy-inspired information using the neural nodes of fully connected layers. This enables substantial nodes to be included in federated averaging (FedAvg), unlike standard FedAvg. In the case of a client model (at an embedded edge cluster), we exploited sparse learning to absorb more features associated with new classes by retraining weak neural nodes. For global model averaging, we used the proposed model selection technique to enhance the performance of traditional FedAvg by up to 20% in terms of global accuracy in the same number of rounds, which, in turn, has increased the convergence in federated rounds. For local sparse learning models, we simulated clients at edge clusters using the Docker container on embedded boards (NVIDIA Jetson Series). To ensure the efficacy of our proposed technique, we rigorously tested our algorithm on various datasets (CIFAR 10, MNIST, OCS lab(driving), UCI-HAR) using different models for image classification (EfficientNet, MobileNetv2, ResNet18) and time-series classification (FCN-LSTM, DepthConv-LSTM), and compared our algorithm with several state-of-the-art methods, such as FedAvg and FedFS. The proposed method outperformed the others in terms of generalizability and assists in converging earlier than existing algorithms in federated rounds.