Weakly-and Semi-Supervised Learning of a Deep Convolutional Network for Semantic Image Segmentation

TLDR

Deep convolutional neural networks trained on large strongly annotated datasets have recently advanced semantic image segmentation. The study investigates training DCNNs for semantic segmentation using weakly annotated data or a mix of few strong and many weak labels. They develop expectation–maximization approaches to train segmentation models under weakly supervised and semi‑supervised regimes. Experiments on PASCAL VOC 2012 demonstrate that the EM‑based methods achieve competitive performance while requiring far less annotation, and the source code is publicly available.

Abstract

Deep convolutional neural networks (DCNNs) trained on a large number of images with strong pixel-level annotations have recently significantly pushed the state-of-art in semantic image segmentation. We study the more challenging problem of learning DCNNs for semantic image segmentation from either (1) weakly annotated training data such as bounding boxes or image-level labels or (2) a combination of few strongly labeled and many weakly labeled images, sourced from one or multiple datasets. We develop Expectation-Maximization (EM) methods for semantic image segmentation model training under these weakly supervised and semi-supervised settings. Extensive experimental evaluation shows that the proposed techniques can learn models delivering competitive results on the challenging PASCAL VOC 2012 image segmentation benchmark, while requiring significantly less annotation effort. We share source code implementing the proposed system at https://bitbucket.org/deeplab/deeplab-public.