2018 · 34 citations · 15 references
EngineeringAerospace EngineeringAutonomous SystemsAerial RoboticsArtificial IntelligenceAvionicsFlight SimulationIntelligent AgentsMission PlanningPolicy LearningReinforcement LearningAutonomous Flight ControlDeep Reinforcement LearningReinforcement Learning For RoboticsSense-and-avoidTraffic ManagementTrainingUnmanned Aerial Vehicle ControlVisual Flight Guidance
Drones are rapidly increasing their activity in the airspace worldwide. This expected growth of the number of drones makes human-based traffic management prohibitive. Avionics systems able to sense-and-avoid obstacles and specially visual flight rules (VFR) traffic are under research. Moreover, to overcome loss-link contingencies, drones have to be able to act autonomously. In this paper we present a drone concept with a full level of autonomy based on Deep Reinforcement Learning (DRL). From the first flight until the accomplishment of its final mission, the drone has no need for a pilot. The only human intervention is the engineer programming the artificial intelligence algorithm used to train and then to control the drone. In this paper we present the preliminary results for an environment which is a realistic flight simulator, and an agent that is a quad-copter drone able to execute 3 actions. The inputs of the agent are the current state and the accumulated reward. Experiments include self-learning periods up to 3 days, followed by one hundred full-autonomous flight tests. Three different DRL algorithms were used to obtain the training models, based in Q-learning reinforcement learning. Results are very promising, with around an 80 percent of test flights reaching the target. In comparison with the results of a human pilot, acting in the same simulated environment and using the same three actions, the DRL methods demonstrated unequal results, depending on the learning algorithm used. We applied some enhancements in the training, with the creation of checkpoints of the training model every time a better solution is found. In a near future we expect to achieve results similar to the performance of a human pilot to support the idea of full-autonomous drones through DRL methods.
15
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver et al. · Nature · 2015 · 30K citations
Reinforcement Learning: An Introduction
Richard S. Sutton, Andy Barto · IEEE Transactions on Neural Networks · 1998 · 27.1K citations
Christopher J. Watkins, Peter Dayan · Machine Learning · 1992 · 9K citations · Full text
TensorFlow: A system for large-scale machine learning
Martı́n Abadi, Paul Barham, Jianmin Chen et al. · arXiv (Cornell University) · 2016 · 8.8K citations · Full text