Federated Contrastive Learning With Feature-Based Distillation for Human Activity Recognition

Abstract

This article proposes a federated contrastive learning with feature-based distillation (FCLFD) framework tailored for human activity recognition (HAR). The FCLFD system integrates a central server with multiple mobile users to address a diverse range of HAR challenges. The framework encompasses two pivotal elements: a contrastive student--teacher (CST) architecture with feature-based distillation and an average weight scheme (AWS). The CST framework facilitates the transfer of comprehensive knowledge from a teacher model to a student model through feature-based distillation and contrastive learning, with both models sharing an identical architecture. Each participating user periodically uploads the weights of its student model to the central server, where the AWS deployed on the server calculates the average weights based on contributions from all connected users. The aggregated weights are then redistributed to each user, who updates their teacher model accordingly. Experimental evaluations demonstrate that when 50 users are connected, the proposed FCLFD scheme obtains the highest <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> values of 89.01 and 94.19, outperforming several state-of-the-art federated learning algorithms on the wireless sensor data mining (WISDM) and PAMAP2 datasets.