Zero Knowledge Clustering Based Adversarial Mitigation in Heterogeneous Federated Learning

Abstract

The simultaneous development of deep learning techniques and Internet of Things (IoT)/Cyber-physical Systems (CPS) technologies has afforded untold possibilities for improving distributed computing, sensing, and data analysis. Among these technologies, federated learning has received increased attention as a privacy-preserving collaborative learning paradigm, and has shown significant potential in IoT/CPS-driven large-scale smart-world systems. At the same time, the vulnerabilities of deep neural networks, especially to adversarial attacks, cannot be overstated and should not be minimized. Moreover, the distributed nature of federated learning makes defense against such adversarial attacks a more challenging problem due to the unavailability of local data and resource heterogeneity. To tackle these challenges, in this paper, we propose ZeKoC, a Zero Knowledge Clustering approach to mitigating adversarial attacks. Particularly, we first formulate the problem of resource-constrained adversarial mitigation. Specifically, noting that a global server has no access to training samples, we reformulate the unsupervised weight clustering problem. Our proposed ZeKoC approach allows the server to automatically split and merge weight clusters for weight selection and aggregation. Theoretical analysis demonstrates that convergence is guaranteed. Further, our experimental results illustrate that, in a non-i.i.d. (i.e., independent and identically distributed) data setting, the proposed ZeKoC approach successfully mitigates general attacks while outperforming state-of-art schemes.