Asymmetric Non-Local Neural Networks for Semantic Segmentation

TLDR

The non‑local module is useful for semantic segmentation but is criticized for high computation and GPU memory usage. This paper proposes an Asymmetric Non‑Local Neural Network for semantic segmentation comprising APNB and AFNB. The network uses APNB, a pyramid‑sampling non‑local block that cuts computation and memory, and AFNB, which fuses multi‑level features with long‑range dependencies to boost performance. It achieves 81.3 mIoU on Cityscapes and, for 256×128 inputs, APNB runs six times faster and uses 28 times less GPU memory than a standard non‑local block. Code is available at https://github.com/MendelXu/ANN.git.

Abstract

The non-local module works as a particularly useful technique for semantic segmentation while criticized for its prohibitive computation and GPU memory occupation. In this paper, we present Asymmetric Non-local Neural Network to semantic segmentation, which has two prominent components: Asymmetric Pyramid Non-local Block (APNB) and Asymmetric Fusion Non-local Block (AFNB). APNB leverages a pyramid sampling module into the non-local block to largely reduce the computation and memory consumption without sacrificing the performance. AFNB is adapted from APNB to fuse the features of different levels under a sufficient consideration of long range dependencies and thus considerably improves the performance. Extensive experiments on semantic segmentation benchmarks demonstrate the effectiveness and efficiency of our work. In particular, we report the state-of-the-art performance of 81.3 mIoU on the Cityscapes test set. For a 256x128 input, APNB is around 6 times faster than a non-local block on GPU while 28 times smaller in GPU running memory occupation. Code is available at: https://github.com/MendelXu/ANN.git.