Self-tuning Gains of a Quadrotor using a Simple Model for Policy Gradient Reinforcement Learning

Abstract

Autonomous flight of Micro Aerial Vehicles (MAVs) faces many challenges in the realm of control.When approaching these problems it is usually advantageous to have an accurate mathematical model of the system to be controlled.However, this is not always possible to obtain due to the complex nature of MAV dynamics, inconsistency of manufacturing processes, and parameter adaptation over time.A model-free control approach such as model-free Reinforcement Learning (RL) requires a large number of real-life trials to learn desired actions.This paper presents the implementation of an approach which utilizes a simple model of a quadrotor system to reduce the number of real-life trials required to find the locally optimal gains.The concept is verified using 2 simulations: one for an F-16 aircraft and the other for a quadrotor.Finally, the approach is applied on a quadrotor's vertical controller to tune the PID gains during a takeoff maneuver of a real AR.drone 2 quadrotor. Nomenclature MDPMarkov decision process π θ policy θ policy parameters ρ performance measure V π value function of policy, π ∇θρ(θ) derivative vector of δρ δθ