Hybrid Robust Control and Reinforcement Learning for Optimal Upset Recovery

Abstract

Air safety and flight asset protection benefit greatly from rapid upset recovery. Autonomous recovery is of particular interest due to a recent significant increase in fielded Unmanned Aerial Vehicles (UAVs). Autonomous recovery challenges include complex nonlinear dynamics and large variation in potential upset conditions. A novel UAV upset recovery system is developed that combines the benefits of robust control with the benefits of intelligent learning techniques. Off-line, Reinforcement Learning (RL) techniques are applied to simulation data to discover recovery strategies that improve upon known strategies. When learning is complete, the strategies are provided to an online component. In the event of an upset, the online component is interrogated to determine the best control decision at each control update until the recovery is complete. The online component is designed to easily make use of the best-known recovery strategies, taking advantage of improved strategies as learning matures. The system architecture is partitioned into two components; one which focuses on recovery from high angular rate upsets and another which focuses on recovery from unusual attitude upsets. The input and output sets for both partitions are compact by design to reduce complexity, thereby ensuring the applicability of RL techniques. Two simulation variants of NASA’s Generic Transport Model (GTM) are used, one for training and initial evaluation and another for robustness testing. The results indicate that the learning process frequently finds improvements to best-known strategies, and that learned recovery strategies are robust to uncertainty.