Efficient Deep Learning for Stereo Matching

TLDR

Convolutional neural networks have achieved high accuracy in stereo estimation, yet existing siamese architectures require about a minute of GPU time per image pair. This paper proposes a matching network that delivers highly accurate stereo results in under one second of GPU time. The network uses a product layer that computes the inner product of siamese feature maps and is trained as a multi‑class classification over all possible disparities. This yields calibrated scores and significantly improves matching performance compared to existing approaches.

Abstract

In the past year, convolutional neural networks have been shown to perform extremely well for stereo estimation. However, current architectures rely on siamese networks which exploit concatenation followed by further processing layers, requiring a minute of GPU computation per image pair. In contrast, in this paper we propose a matching network which is able to produce very accurate results in less than a second of GPU computation. Towards this goal, we exploit a product layer which simply computes the inner product between the two representations of a siamese architecture. We train our network by treating the problem as multi-class classification, where the classes are all possible disparities. This allows us to get calibrated scores, which result in much better matching performance when compared to existing approaches.