Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials

TLDR

State‑of‑the‑art multi‑class image segmentation uses CRFs over pixels or regions, but pixel‑level models have been limited to sparse graphs due to their size. The paper aims to develop a highly efficient approximate inference algorithm for fully connected CRFs with Gaussian edge potentials. The authors propose an approximate inference method that exploits Gaussian kernel combinations to efficiently handle the billions of edges in fully connected pixel‑level CRFs. Experiments show that fully connected pixel‑level CRFs significantly improve segmentation and labeling accuracy.

Abstract

Most state-of-the-art techniques for multi-class image segmentation and labeling use conditional random fields defined over pixels or image regions. While region-level models often feature dense pairwise connectivity, pixel-level models are considerably larger and have only permitted sparse graph structures. In this paper, we consider fully connected CRF models defined on the complete set of pixels in an image. The resulting graphs have billions of edges, making traditional inference algorithms impractical. Our main contribution is a highly efficient approximate inference algorithm for fully connected CRF models in which the pairwise edge potentials are defined by a linear combination of Gaussian kernels. Our experiments demonstrate that dense connectivity at the pixel level substantially improves segmentation and labeling accuracy.