Deep learning‐based panoptic segmentation: Recent advances and perspectives

Abstract

Abstract In recent years, panoptic segmentation has drawn increasing amounts of attention, leading to the rapid emergence of numerous related algorithms. A variety of deep neural networks have been used more frequently for panoptic segmentation, which is motivated by the significant success of deep learning methods in other tasks. This article presents a comprehensive exploration of panoptic segmentation, focusing on the analysis and understanding of RGB image data. Initially, the authors introduce the background of panoptic segmentation, including deep learning models and image segmentation. Then, the authors thoroughly cover a variety of panoptic segmentation‐related topics, such as datasets connected to the field, evaluation metrics, panoptic segmentation models, and derived subfields based on panoptic segmentation. Finally, the authors examine the difficulties and possibilities in this area and identify its future paths.