Incremental learning of gestures by imitation in a humanoid robot

TLDR

The study proposes an incremental gesture learning framework for a humanoid robot, comparing two incremental training methods to a batch baseline and demonstrating that multimodal teaching can efficiently train basketball officials' signals. The method first records the user’s arm and head motion with sensors, projects it into a latent space, and encodes it in a Gaussian Mixture Model; then the user refines the gesture through kinesthetic teaching, and the authors compare two incremental training procedures to a batch baseline. Active teaching with the human teacher in the loop enables efficient transfer of essential gesture characteristics, and multimodal approaches effectively train basketball officials' signals on the HOAP‑3 robot.

Abstract

We present an approach to teach incrementally human gestures to a humanoid robot. By using active teaching methods that puts the human teacher "in the loop" of the robot's learning, we show that the essential characteristics of a gesture can be efficiently transferred by interacting socially with the robot. In a first phase, the robot observes the user demonstrating the skill while wearing motion sensors. The motion of his/her two arms and head are recorded by the robot, projected in a latent space of motion and encoded bprobabilistically in a Gaussian Mixture Model (GMM). In a second phase, the user helps the robot refine its gesture by kinesthetic teaching, i.e. by grabbing and moving its arms throughout the movement to provide the appropriate scaffolds. To update the model of the gesture, we compare the performance of two incremental training procedures against a batch training procedure. We present experiments to show that different modalities can be combined efficiently to teach incrementally basketball officials' signals to a HOAP-3 humanoid robot.