Multi-Scale Context Aggregation by Dilated Convolutions

TLDR

Semantic segmentation models are adapted from image‑classification networks, yet dense prediction differs structurally from classification. The study introduces a new convolutional module tailored for dense prediction. The module employs dilated convolutions to aggregate multi‑scale context while preserving resolution and achieving exponential receptive field growth. The context module improves accuracy of state‑of‑the‑art segmentation, and simplifying adapted classification networks further boosts performance.

Abstract

State-of-the-art models for semantic segmentation are based on adaptations of convolutional networks that had originally been designed for image classification. However, dense prediction and image classification are structurally different. In this work, we develop a new convolutional network module that is specifically designed for dense prediction. The presented module uses dilated convolutions to systematically aggregate multi-scale contextual information without losing resolution. The architecture is based on the fact that dilated convolutions support exponential expansion of the receptive field without loss of resolution or coverage. We show that the presented context module increases the accuracy of state-of-the-art semantic segmentation systems. In addition, we examine the adaptation of image classification networks to dense prediction and show that simplifying the adapted network can increase accuracy.