Apprenticeship Bootstrapping

Abstract

Apprenticeship learning is a learning scheme based on the direct imitation of humans. Inverse reinforcement learning is used to learn a reward function from human data. Coupling Inverse reinforcement learning with reinforcement learning has demonstrated production of human-competitive policies. However, obtaining human subjects with the right level of skills for complex tasks can be a challenge. We propose a new learning scheme called Apprenticeship Bootstrapping to learn a composite task using human demonstrations on sub-tasks. The scenario is a ground-air interaction task with an Unmanned Aerial Vehicle that needs to maintain 3 autonomous Unmanned Ground Vehicles within range of an imaging sensor. For validation, we show that the bootstrapped policy performs as good as a policy learnt from a human performing the composite task. The method offers a clear advantage when skilled humans are available for simpler tasks that form the building blocks for a more complex task, where availability of experts is limited.