IEEE Transactions on Wireless Communications · 2024 · 56 citations · 27 references
ComputingAerospace EngineeringMachine LearningMarkov Decision ProcessesQueuing TheoryInterference ManagementMulti-agent SystemsReinforcement LearningSatellite Constellation ArchitectureDeep Reinforcement LearningHeterogeneous NetworksInterferenceInterference AvoidanceMulti-agent LearningMulti-agent Reinforcement LearningMulti-agent SchedulingSpectrum Efficiency
Non-geostationary orbit (NGSO) constellations enabled by beam hopping (BH) technology are characterized by wide coverage and high spectrum efficiency. However, how to efficiently schedule multi-satellite beam resources to satisfy the heterogeneous and uneven terrestrial traffic demands remains a huge challenge for satellite operators. This paper proposes a satellite-terrestrial coordinated multi-satellite BH scheduling framework, where the complex multi-satellite BH problem is formulated into a long-term and a short-term subproblems. The long-term subproblem is cell-satellite association problem, which is solved by a low-complexity iterative algorithm executed in network operation control center (NOCC) to minimize the traffic load gap among satellites while considering the interference avoidance. The short-term subproblem is multi-satellite traffic-driven BH problem and we propose a multi-agent deep reinforcement learning (MADRL) architecture where each satellite can cooperatively make real-time BH decisions using the well-trained model by QMIX algorithm to adapt to time-varying and heterogeneous traffic. Simulation results demonstrate that the traffic load gap and network delay have been reduced by 70% and 50% respectively compared with non-load-balancing scheme. Besides, the proposed algorithm outperforms other benchmarks in terms of the network throughput under various traffic load cases and the average network delay is kept within 4 ms. Furthermore, the proposed QMIX-BH can be applied to real-time scheduling since the execution time is less than 1 ms.
27
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Jun‐Young Chung, Çağlar Gülçehre, Kyunghyun Cho et al. · arXiv (Cornell University) · 2014 · 10.8K citations · Full text
Deep Reinforcement Learning with Double Q-Learning
Hado van Hasselt, Arthur Guez, David Silver · 2016 · 6K citations · Full text