Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information

TLDR

The paper addresses accurate stereo matching, focusing on object boundaries, robustness to recording or illumination changes, and computational efficiency. The authors introduce two novel contributions. They propose a semi‑global matching method that uses pixelwise mutual‑information matching with an approximate global smoothness constraint, and extend it to multi‑baseline stereo images. The method detects occlusions and computes sub‑pixel disparities, achieves near‑intensity‑matching speed with a hierarchical mutual‑information approach, and offers a linear‑time global cost approximation that runs in about one second on typical images.

Abstract

This paper considers the objectives of accurate stereo matching, especially at object boundaries, robustness against recording or illumination changes and efficiency of the calculation. These objectives lead to the proposed semi-global matching method that performs pixelwise matching based on mutual information and the approximation of a global smoothness constraint. Occlusions are detected and disparities determined with sub-pixel accuracy. Additionally, an extension for multi-baseline stereo images is presented. There are two novel contributions. Firstly, a hierarchical calculation of mutual information based matching is shown, which is almost as fast as intensity based matching. Secondly, an approximation of a global cost calculation is proposed that can be performed in a time that is linear to the number of pixels and disparities. The implementation requires just 1 second on typical images.