Joint Mean-Field Game and Multiagent Asynchronous Advantage Actor-Critic for Edge Intelligence-Based IoT Malware Propagation Defense

Abstract

Defending Edge intelligence-based Internet of Things (EIoT) systems by controlling malware propagation has become a critical issue. Herein, we meet new challenges brought by multiple attackers and defenders for effective mitigation of malware propagation in EIoT. To explore the dynamic changes of malware propagation, a state transition diagram of IoT nodes is proposed to describe the mutual changes of five states: infected, active, dormant, isolated, and hardened, and differential equations for each state are established. We then build a mean-field game model representing the interactions between multiattackers and multidefenders in the EIoT malware propagation defense environment. We further convert the problem of solving the game into an MDP (Markov Decision Process) and propose a distributed algorithm called MFGA3C (Mean-Field Game-based Asynchronous Advantage Actor-Critic) that combines mean-field game and multiagent asynchronous advantage actor-critic to learn the optimal defense policy. Finally, we compare our algorithm MFGA3C with other benchmark algorithms in the EIoT malware propagation countermeasure environment. Experimental results show that our MFGA3C algorithm dominates the average reward, total reward, and the number of successful defenses under three typical EIoT-environmental conditions, which indicates that MFGA3C faster and more robustly learns the optimal malware propagation defense policy, contributing to protecting EIoT systems.