Pick and Place Objects in a Cluttered Scene Using Deep Reinforcement Learning

Abstract

This paper presents a robotic grasp-to-place system that has the capability of grasping objects in sparse and cluttered environments. The key feature of the system is that it handles both primitive actions of picking and placing of objects with an explicit framework using raw RGB-D images. Thus, the contribution of this paper is to model such a complete manipulation system with reasonable computational complexity. To achieve this, the camera captured RGB images of the scene, and 3d point cloud information are used to generate heightmaps at the robot grasp-workspace. The heightmap is rotated by 36 different angles before feeding into the network in order to generate a set of 36 pixel-wise Q-value maps, which is then passed into a Dense Network (DenseNet) to generate predictions of Q-values. Q-values are a measure of future expected reward in the formula of Q-learning from reinforcement learning. This effectively gives us values for predictions for 36 different grasping angles for every visible location in the robot grasp-workspace. In the simulation, exhaustive experimental results demonstrate that our framework successfully grasp objects with a grasp success rate and grasp efficiency, at almost 80~95% for both. As for place task, our framework successfully placed objects with a place success rate to at least 90 % through all test-cases.