Multiobjective Reinforcement Learning for Cognitive Satellite Communications Using Deep Neural Network Ensembles

TLDR

Future spacecraft communication subsystems could benefit from AI‑controlled software‑defined radios, building on prior work that constrains low‑performance decisions via virtual environment exploration. The paper proposes a radio resource allocation algorithm that uses multiobjective reinforcement learning and neural network ensembles to balance resource use and mission goals. The algorithm models the continuous multidimensional state–action space with a fixed‑size memory mapping and decouples actions from states to enable online learning, performance monitoring, and resource‑allocation prediction. Simulations demonstrate the algorithm’s performance across various mission profiles and provide accuracy benchmarks, and the approach has been integrated into NASA Glenn Research Center’s SCaN Testbed radios aboard the International Space Station.

Abstract

Future spacecraft communication subsystems will potentially benefit from software-defined radios controlled by artificial intelligence algorithms. In this paper, we propose a novel radio resource allocation algorithm leveraging multiobjective reinforcement learning and artificial neural network ensembles able to manage available resources and conflicting mission-based goals. The uncertainty in the performance of thousands of possible radio parameter combinations and the dynamic behavior of the radio channel over time producing a continuous multidimensional state–action space requires a fixed-size memory continuous state–action mapping instead of the traditional discrete mapping. In addition, actions need to be decoupled from states in order to allow for online learning, performance monitoring, and resource allocation prediction. The proposed approach leverages the authors' previous research on constraining decisions predicted to have poor performance through "virtual environment exploration." The simulation results show the performance for different communication mission profiles, and accuracy benchmarks are provided for the future research reference. The proposed approach constitutes part of the core cognitive engine proof-of-concept delivered to the NASA John H. Glenn Research Center's SCaN Testbed radios on-board the International Space Station.