Multiple Access via Curriculum Multitask HAPPO Based on Dynamic Heterogeneous Wireless Network

Abstract

With the development of wireless communication systems, the large-scale deployment of Internet of Things (IoT) devices becomes popular. Due to limited energy, the multiple access approaches without carrier sensing requirement are widely deployed in IoT devices, including Aloha and time-division multiple access (TDMA). However, these approaches encounter the transmission inefficiency issue, especially in dynamic heterogeneous networks comprising nodes with diverse protocols and varying numbers and transmission configurations over time. In this article, combining curriculum learning (CL) and multitask reinforcement learning (MTRL), we propose the curriculum multitask heterogeneous-agent proximal policy optimization (CMHA) algorithm to improve the throughput performance while guaranteeing fairness in dynamic heterogeneous networks. We introduce the elastic weight consolidation (EWC) in the CMHA to further enhance generalization capacity, which can better address the challenging MTRL problem in dynamic heterogeneous networks. Combining the monotonic improvement feature of heterogeneous-agent proximal policy optimization (HAPPO) and the generalization capacity of EWC, the proposed CMHA can achieve a nearly monotonic improvement in all possible scenarios. The simulations show that the CMHA <xref ref-type="algorithm" rid="alg1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">algorithm 1</xref> ) has sufficient generalization capacity for massive scenarios in dynamic heterogeneous networks; 2) can significantly enhance the network throughput; and 3) can guarantee the fairness of both agents and heterogeneous nodes.