Semantic Image Segmentation via Deep Parsing Network

TLDR

The study proposes a semantic image segmentation approach that enriches Markov Random Fields with high‑order relations and mixed label contexts. The authors introduce Deep Parsing Network, a CNN that models unary terms and approximates mean‑field inference for pairwise terms, enabling deterministic end‑to‑end MRF optimization in a single forward pass. On PASCAL VOC 2012, DPN attains a new state‑of‑the‑art 77.5 % accuracy, outperforming prior CNN–MRF methods while requiring only one mean‑field iteration and enabling GPU parallelization.

Abstract

This paper addresses semantic image segmentation by incorporating rich information into Markov Random Field (MRF), including high-order relations and mixture of label contexts. Unlike previous works that optimized MRFs using iterative algorithm, we solve MRF by proposing a Convolutional Neural Network (CNN), namely Deep Parsing Network (DPN), which enables deterministic end-to-end computation in a single forward pass. Specifically, DPN extends a contemporary CNN architecture to model unary terms and additional layers are carefully devised to approximate the mean field algorithm (MF) for pairwise terms. It has several appealing properties. First, different from the recent works that combined CNN and MRF, where many iterations of MF were required for each training image during back-propagation, DPN is able to achieve high performance by approximating one iteration of MF. Second, DPN represents various types of pairwise terms, making many existing works as its special cases. Third, DPN makes MF easier to be parallelized and speeded up in Graphical Processing Unit (GPU). DPN is thoroughly evaluated on the PASCAL VOC 2012 dataset, where a single DPN model yields a new state-of-the-art segmentation accuracy of 77.5%.