DOFMS: DRL-Based Out-of-Order Friendly Multipath Scheduling in Mobile Heterogeneous Networks

Abstract

Multipath transmission brings strong bandwidth aggregation capability for services in wireless networks. Nonetheless, the heterogeneous nature of paths and the motion of terminals results in varying transmission delays, leading to out-of-order (OFO) delivery and transmission quality decrease. Traditional algorithms, limited in their scope, fail to strike a balance between high bandwidth and low OFO extent. Recent studies have focused on utilizing learning algorithms to find a multi-performance joint optimal transmission strategy. In light of this, this paper proposes a framework called DRL-based OFO-Friendly Multipath Scheduling (DOFMS) to ensure high bandwidth and low OFO extent transmission in mobile heterogeneous networks. In particular, the framework introduces a novel OFO evaluation index to assess the degree of OFO more accurately. To achieve elastic scheduling, the framework employs the Double Deep Q Network (DDQN) to dynamically regulate the scheduling ratio. Recognizing the dynamic and unpredictable nature of path delays, an asynchronous module is introduced to enhance learning accuracy. Experimental results demonstrate that the framework reduces the OFO rate by 25% compared to traditional bandwidth aggregation algorithms, while maintaining low bandwidth and packet loss rates. Furthermore, compared to conventional OFO avoidance algorithms, the framework improves bandwidth by 4% and reduces fluctuation by 90%.