Combining Electromyography and Fiducial Marker Based Tracking for Intuitive Telemanipulation with a Robot Arm Hand System

Abstract

Teleoperation and telemanipulation have since the early years of robotics found use in a wide range of applications, including exploration, maintenance, and response in remote or hazardous environments, healthcare, and education settings. As the capabilities of robot manipulators grow, so does the control complexity and the remote execution of intricate manipulation tasks still remains challenging for the user. This paper proposes an intuitive telemanipulation framework based on electromyography (EMG) and fiducial marker based tracking that can be used with a dexterous robot arm hand system. The EMG subsystem captures the myoelectric activations of the user during the execution of specific hand postures and gestures and translates them into the desired grasp type for the robot hand. The pose of the tracked fiducial marker is used as a task-space goal for the robot end-effector. The system performance is experimentally validated in a remote operation setting, where the system successfully performs a telemanipulation task.