IEEE Journal on Selected Areas in Communications · 2018 · 139 citations · 20 references
ComputingAerospace EngineeringComputer ScienceArtificial IntelligenceMachine LearningOptimizationCognitive RadioExploration-exploitation TradeoffMission PlanningMulti-objective Markov Decision ProcessesReinforcement LearningSpace ExplorationCognitive Radio Resource AllocationDeep Reinforcement LearningMission-based Decision MakingMultiobjective Reinforcement LearningPerformance MonitoringRadio Resource AllocationRadio Resource ManagementSoftware Defined Radio
Future spacecraft communication subsystems could benefit from AI‑controlled software‑defined radios, building on prior work that constrains low‑performance decisions via virtual environment exploration. The paper proposes a radio resource allocation algorithm that uses multiobjective reinforcement learning and neural network ensembles to balance resource use and mission goals. The algorithm models the continuous multidimensional state–action space with a fixed‑size memory mapping and decouples actions from states to enable online learning, performance monitoring, and resource‑allocation prediction. Simulations demonstrate the algorithm’s performance across various mission profiles and provide accuracy benchmarks, and the approach has been integrated into NASA Glenn Research Center’s SCaN Testbed radios aboard the International Space Station.
Future spacecraft communication subsystems will potentially benefit from software-defined radios controlled by artificial intelligence algorithms. In this paper, we propose a novel radio resource allocation algorithm leveraging multiobjective reinforcement learning and artificial neural network ensembles able to manage available resources and conflicting mission-based goals. The uncertainty in the performance of thousands of possible radio parameter combinations and the dynamic behavior of the radio channel over time producing a continuous multidimensional state–action space requires a fixed-size memory continuous state–action mapping instead of the traditional discrete mapping. In addition, actions need to be decoupled from states in order to allow for online learning, performance monitoring, and resource allocation prediction. The proposed approach leverages the authors' previous research on constraining decisions predicted to have poor performance through "virtual environment exploration." The simulation results show the performance for different communication mission profiles, and accuracy benchmarks are provided for the future research reference. The proposed approach constitutes part of the core cognitive engine proof-of-concept delivered to the NASA John H. Glenn Research Center's SCaN Testbed radios on-board the International Space Station.
20
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver et al. · Nature · 2015 · 30K citations
Reinforcement Learning: An Introduction
IEEE Transactions on Neural Networks · 2005 · 25.8K citations
Mastering the game of Go with deep neural networks and tree search
David Silver, Aja Huang, Chris J. Maddison et al. · Nature · 2016 · 15.8K citations
Reinforcement Learning: An Introduction
Jeffrey D. Johnson, Jinghong Li, Zengshi Chen · Neurocomputing · 2000 · 8.7K citations