Real-time precise object segmentation using a pixel-wise coarse-fine method with deep learning for automated manufacturing

Abstract

The Fourth Industrial Revolution has rapidly increased the demand, beyond the pace of industrial automation, for artificial intelligence (AI) and robots to undertake sophisticated tasks such as assembly and packaging without the use of feed devices. For commercial use, suitable technologies for stable real-time detection and precise segmentation are deemed essential. Particularly, because AI is used in an embedded environment, the model and algorithm should be lightweight. To satisfy the industrial requirements and solve related difficulties, this paper proposes a precision object segmentation method to accurately detect objects moving on a conveyor belt without using feed devices. A new backbone network was designed to reinforce the feature extraction layers using convolutional blocks, and it was added to the object detection component. To maintain lightweight compactness and complement feature mapping, a multi-level pooling layer was included. In addition, a loss function that focuses on difficult samples and a data augmentation method for learning were used to improve the performance without affecting the model architecture. In the segmentation step, the authors extracted and analyzed pixel-wise features to acquire more pertinent vertex candidates and utilized an intensity-difference search algorithm to select the optimal outline component among the candidates for precise segmentation. Finally, the posture information of the object was estimated using the object’s geometric features to estimate the grasping point. The experimental results confirmed the improved precision and accuracy of the proposed method in object-detection and segmentation, demonstrating that the model is ideal for our manufacturing dataset.