Multi-Level Fusion of the Multi-Receptive Fields Contextual Networks and Disparity Network for Pairwise Semantic Stereo

Abstract

In this paper, we propose a multi-level fusion framework to address the pairwise semantic stereo issue. For disparity estimation, we adopt the pyramid stereo matching network. For semantic segmentation, the single segmentation network is proposed with respect to the left image, along with the disparity fusion segmentation network for the combination of semantic features and disparity features. Specifically, the multi-receptive fusion block is designed and employed to fully extract and fuse the contextual information. Finally, the refined segmentation result is obtained via yet another fusion of the multi-model results. The proposed method achieved a mean intersection over union (mIoU) of 79.05%, an average endpoint error (EPE) of 1.3966, and an mIoU-3 of 77.75%, ranking first in the Pairwise Semantic Stereo Challenge of the 2019 IEEE GRSS Data Fusion Contest [1],[2].