Detecting layered structures of partially occluded objects for bin picking

Abstract

When robots engage in bin picking of multiple objects, a failure in grasping partially occluded objects may occur because other objects may overlap the desired ones. Therefore, the layered structure of objects needs to be detected, and the picking order needs to be established. In this paper, we propose a new dataset that evaluates not only the area of objects but also the layered structures of objects. In this dataset, three tasks are targeted: object detection, semantic segmentation, and segmentation of occluded areas for bin picking of multiple objects. The dataset, called “the Amazon Robotics Challenge (ARC) Multi-task Dataset” contains 1,500 RGB images and depth images, including all scenes containing bounding box labels, semantic segmentation labels, and occluded area labels. This enables representing the layered structure of overlapped objects with a tree structure. A benchmark of the ARC multi-task dataset demonstrated that occluded areas could be segmented using a Mask regional convolutional neural network (R-CNN) and that layered structures of objects could be predicted. Our dataset is available at the following URL:http://mprg.jp/research/arc_dataset_2017_e.