Multi-class image segmentation using conditional random fields and global classification

TLDR

Semantic image segmentation requires integrating local and global features to predict accurate segment labels. The authors propose a multi‑class segmentation method that combines a Conditional Random Field (CRF) for local features with a global image classifier. The method employs a CRF built on unsupervised multi‑scale pre‑segmentation of image patches, treating patch labels as random variables, and constrains the CRF using the classifier’s output. The approach was evaluated on a standard semantic segmentation dataset, demonstrating its performance relative to baseline methods.

Abstract

A key aspect of semantic image segmentation is to integrate local and global features for the prediction of local segment labels. We present an approach to multi-class segmentation which combines two methods for this integration: a Conditional Random Field (CRF) which couples to local image features and an image classification method which considers global features. The CRF follows the approach of Reynolds & Murphy (2007) and is based on an unsupervised multi scale pre-segmentation of the image into patches, where patch labels correspond to the random variables of the CRF. The output of the classifier is used to constraint this CRF. We demonstrate and compare the approach on a standard semantic segmentation data set.