Least Square Policy Iteration for IBVS based Dynamic Target Tracking

Abstract

This paper delves into the problem of tracking a maneuvering target based on only vision based feedback namely Image Based Visual Servoing (IBVS). In the absence of a GPS or any other external sensor, vision proves to be a valuable source of information about the environment. However it is difficult to perform the tracking using only a monocular vision due to the absence of depth measurement. This restricts the use of traditional IBVS methods as they rely on the interaction matrix which is sensitive to various camera parameters and depth estimate. We thus solve this problem through a learning based approach and model it as Markov decision process, on which we apply a Reinforcement Learning technique. Least Square Policy Iteration (LSPI) learns the optimal control policies required to keep following the target drone while maintaining a fixed distance from it. The performance of the proposed algorithm is tested and simulated in Gazebo environment.