Repeatable Folding Task by Humanoid Robot Worker Using Deep Learning

TLDR

The study proposes a machine‑learning method to develop a humanoid robot capable of performing production line tasks. The method employs a real‑time teleoperation interface with a head‑mounted display to collect task data, then trains a two‑phase deep learning model—a convolutional autoencoder followed by a time‑delay neural network—to learn task dynamics, and evaluates the model on the Nextage Open robot performing an object‑folding task with 35 trained and 5 untrained sequences. Online testing of the trained model achieved a 77.8% success rate on the object‑folding task.

Abstract

We propose a practical state-of-the-art method to develop a machine-learning-based humanoid robot that can work as a production line worker. The proposed approach provides an intuitive way to collect data and exhibits the following characteristics: task performing capability, task reiteration ability, generalizability, and easy applicability. The proposed approach utilizes a real-time user interface with a monitor and provides a first-person perspective using a head-mounted display. Through this interface, teleoperation is used for collecting task operating data, especially for tasks that are difficult to be applied with a conventional method. A two-phase deep learning model is also utilized in the proposed approach. A deep convolutional autoencoder extracts images features and reconstructs images, and a fully connected deep time delay neural network learns the dynamics of a robot task process from the extracted image features and motion angle signals. The "Nextage Open" humanoid robot is used as an experimental platform to evaluate the proposed model. The object folding task utilizing with 35 trained and 5 untrained sensory motor sequences for test. Testing the trained model with online generation demonstrates a 77.8% success rate for the object folding task.