Augmenting CRFs with Boltzmann Machine Shape Priors for Image Labeling

Andrew Kae, Kihyuk Sohn, Honglak Lee, Erik Learned-Miller

2013 · 187 citations · 28 references

Concepts

TL;DR

CRFs excel at modeling local interactions among adjacent image regions but struggle with complex, long‑range dependencies, whereas RBMs can capture global shape priors. The study proposes a hybrid CRF–RBM model that leverages both local and global modeling to achieve state‑of‑the‑art image labeling. The hybrid architecture augments a CRF baseline with an RBM that supplies a global shape bias, and its performance is evaluated on facial parts from the Labeled Faces in the Wild dataset. The hybrid model outperforms the CRF alone in both quantitative metrics and visual quality, and its RBM hidden units correspond to interpretable face attributes learned without attribute‑level supervision.

Abstract

Conditional random fields (CRFs) provide powerful tools for building models to label image segments. They are particularly well-suited to modeling local interactions among adjacent regions (e.g., super pixels). However, CRFs are limited in dealing with complex, global (long-range) interactions between regions. Complementary to this, restricted Boltzmann machines (RBMs) can be used to model global shapes produced by segmentation models. In this work, we present a new model that uses the combined power of these two network types to build a state-of-the-art labeler. Although the CRF is a good baseline labeler, we show how an RBM can be added to the architecture to provide a global shape bias that complements the local modeling provided by the CRF. We demonstrate its labeling performance for the parts of complex face images from the Labeled Faces in the Wild data set. This hybrid model produces results that are both quantitatively and qualitatively better than the CRF alone. In addition to high-quality labeling results, we demonstrate that the hidden units in the RBM portion of our model can be interpreted as face attributes that have been learned without any attribute-level supervision.

References

28