A Deep Reinforcement Learning based Routing Scheme for LEO Satellite Networks in 6G

Abstract

With the increasing demands of global communication, integrating the low-earth-orbit (LEO) satellite networks (LSNs) with the existing terrestrial networks can significantly extend the wireless coverage in 6G networks. Since the deployment cost of LEO satellites is extremely high, an energy-efficient routing scheme should be designed to prolong the lifetime of LEO satellites while satisfying the end-to-end delay constraint of each data flow. Thus, the routing problem significantly affects the performance of an LSN. In this paper, we formulate this problem as a nonlinear programming problem, which aims to minimize the energy consumption of an LSN caused by data transmission among LEO satellites while satisfying the end-to-end delay constraint of each data flow. We propose a centralized LSN (CLSN) architecture that utilizes a routing manager in a medium-earth-orbit (MEO) satellite to monitor the condition of LEO satellites and intelligently decide the routing path for each routing request event from LEO satellites. We further propose an energy-efficient event-driven deep reinforcement learning (DRL), Deep Deterministic Policy Gradient (DDPG), enhanced Dijkstra’s algorithm (EEDRL-Dijkstra) in the routing manager to deal with the stochastic routing request event arrivals so as to achieve long-term optimization of the energy consumption performance of an LSN while satisfying the end-to-end delay constraint of each data flow. The simulation results show that in the proposed CLSN architecture, the proposed EEDRL-Dijkstra significantly improves the energy consumption performance as compared to Dijkstra’s algorithm and random routing. Both the proposed EEDRL-Dijkstra and Dijkstra’s algorithm can satisfy the end-to-end delay constraint of each data flow.