Optimizing Keypoint-based Single-Shot Camera-to-Robot Pose Estimation through Shape Segmentation

Abstract

We introduce an optimization method for recent approaches on keypoint-based pose estimation of robotic manipulators utilizing monocular images. The method takes into account the segmented shape of the robot using Convolutional Neural Networks and a keypoint refinement through a set of score values. To this end, the primal 2D keypoint detection is exploited as an initial guess for further shape-based keypoint adjustments. Afterwards, the overall methods incorporates a perspective-n-point algorithm using 3D point correspondences that are derived by forward kinematics. We hereby complement an existing public dataset with annotated segmentations of a Universal Robot UR5 manipulator. The evaluation of the optimization approach shows clearly that noise on the initial key-point detection can be suppressed and minimized. Furthermore, the overall success rate of the perspective transformation can be enhanced towards more than 90%. Thus, the overall methods is applicable for single-shot pose estimation. The evaluation results also show a significant reduction of the standard deviation of the resulting pose estimation. Consequently, the proposed optimization positively affects applicability and precision.