HoloSeg: An Efficient Holographic Segmentation Network for Real-time Scene Parsing

Abstract

Real-time semantic segmentation is a crucial but challenging dense prediction task for scene parsing. However, the existing CNN-based methods commonly bias the model in favor of speed-boosting compromising spatial resolution due to business requirements and hardware constrains, which impedes the high-accuracy segmentation result. To address the dilemma, we provide a novel Holographic Segmentation Network (HoloSeg), which presents a strong ability of comprehensive information preservation and extraction, and achieves a better trade-off between speed and accuracy. We first design a Lossless Sample Pair (LSP) without any stride for early spatial preservation and later resolution recovery while modeling long-range context dependence. Then, we propose Distributed Pyramid Learning (DPL) to efficiently extract multiscale features and saves a lot of computation. Finally, we propose Resolution Fusion and Restoration (RFR) to fuse multi-level semantic representations across stages and generate output without decoder. Without bells and whistles, HoloSeg achieves state-of-the-art performance on the Cityscapes benchmark which reports 76.24% mIoU at 231 FPS. Code is available online: https://github.com/LiShuTJ/HoloSeg.