arXiv (Cornell University) · 2020 · 16 citations · 9 references
ComputingAerospace EngineeringComputer ScienceArtificial IntelligenceMachine LearningMarkov Decision ProcessesAircraft ControlDeep LearningEnsemble LearningPolicy LearningReinforcement LearningDeep Reinforcement LearningGuidanceMulti-agent LearningMulti-agent Reinforcement LearningMulti-agent SimulationTraffic Congestion
Air traffic control is an example of a highly challenging operational problem that is readily amenable to human expertise augmentation via decision support technologies. In this paper, we propose a new intelligent decision making framework that leverages multi-agent reinforcement learning (MARL) to dynamically suggest adjustments of aircraft speeds in real-time. The goal of the system is to enhance the ability of an air traffic controller to provide effective guidance to aircraft to avoid air traffic congestion, near-miss situations, and to improve arrival timeliness. We develop a novel deep ensemble MARL method that can concisely capture the complexity of the air traffic control problem by learning to efficiently arbitrate between the decisions of a local kernel-based RL model and a wider-reaching deep MARL model. The proposed method is trained and evaluated on an open-source air traffic management simulator developed by Eurocontrol. Extensive empirical results on a real-world dataset including thousands of aircraft demonstrate the feasibility of using multi-agent RL for the problem of en-route air traffic control and show that our proposed deep ensemble MARL method significantly outperforms three state-of-the-art benchmark approaches.
9
Ray RLLib: A Composable and Scalable Reinforcement Learning Library
Eric Liang, Richard Liaw, Robert Nishihara et al. · arXiv (Cornell University) · 2017 · 109 citations · Full text