Data-Driven Body-Machine Interface for Drone Intuitive Control through Voice and Gestures

Abstract

Aerial drones can be used for a number of monitoring and control applications. Most of existing drone control platforms are quite primitive in terms of body-machine interface. They are usually a variation of a hand-held remote controller or ground control station. However, in a number of line-of-sight scenarios it would be more convenient to use the human gestures and voice for the drone control. In this work, we present an approach for instantaneous control of drones based on human voice and gestures. The proposed solution includes wearable sensors and embedded artificial intelligence. We use a microphone and an Inertial Measurement Unit (IMU) for capturing the human voice and the hand movements. Primary control is implemented by a voice recognition unit based on Recurrent Neural Network (RNN) while the secondary control is implemented by the gesture recognition system based on Convolutional Neural Network (CNN). For implementing the embedded intelligence, we use a low-power embedded system with a graphical processing unit able to run pre-trained neural networks on board of the drone. As a result, the system can perform different speech and gesture recognition tasks real-time.