Multiagent DDPG-Based Deep Learning for Smart Ocean Federated Learning IoT Networks

Abstract

This article proposes a novel multiagent deep reinforcement learning-based algorithm which can realize federated learning (FL) computation with Internet-of-Underwater-Things (IoUT) devices in the ocean environment. According to the fact that underwater networks are relatively not easy to set up reliable links by huge fading compared to wireless free-space air medium, gathering all training data for conducting centralized deep learning training is not easy. Therefore, FL-based distributed deep learning can be a suitable solution for this application. In this IoUT network (IoUT-Net) scenario, the FL system needs to construct a global learning model by aggregating the local model parameters that are obtained from individual IoUT devices. In order to reliably deliver the parameters from IoUT devices to a centralized FL machine, base station like devices are needed. Therefore, a joint cell association and resource allocation (JCARA) method is required and it is designed inspired by multiagent deep deterministic policy gradient (MADDPG) to deal with distributed situations and unexpected time-varying states. The performance evaluation results show that our proposed MADDPG-based algorithm achieves 80% and 41% performance improvements than the standard actor–critic and DDPG, respectively, in terms of the downlink throughput.