Stereo Processing by Semiglobal Matching and Mutual Information

TLDR

The paper proposes the Semi‑Global Matching (SGM) stereo method for efficient and accurate depth estimation. SGM computes a pixel‑wise Mutual‑Information matching cost, enforces smoothness with a global cost, and performs fast path‑wise optimization from multiple directions, while also handling occlusions, subpixel refinement, multi‑baseline matching, outlier removal, structured‑environment issues, gap interpolation, large‑image processing, and disparity fusion via orthographic projection. SGM achieves state‑of‑the‑art accuracy—especially with subpixel precision—while running in 1–2 s on typical images, thanks to its linear complexity, MI‑based cost robustness to radiometric changes, and demonstrated effectiveness on large aerial and pushbroom reconstructions.

Abstract

This paper describes the Semi-Global Matching (SGM) stereo method. It uses a pixelwise, Mutual Information based matching cost for compensating radiometric differences of input images. Pixelwise matching is supported by a smoothness constraint that is usually expressed as a global cost function. SGM performs a fast approximation by pathwise optimizations from all directions. The discussion also addresses occlusion detection, subpixel refinement and multi-baseline matching. Additionally, postprocessing steps for removing outliers, recovering from specific problems of structured environments and the interpolation of gaps are presented. Finally, strategies for processing almost arbitrarily large images and fusion of disparity images using orthographic projection are proposed.A comparison on standard stereo images shows that SGM is among the currently top-ranked algorithms and is best, if subpixel accuracy is considered. The complexity is linear to the number of pixels and disparity range, which results in a runtime of just 1-2s on typical test images. An in depth evaluation of the Mutual Information based matching cost demonstrates a tolerance against a wide range of radiometric transformations. Finally, examples of reconstructions from huge aerial frame and pushbroom images demonstrate that the presented ideas are working well on practical problems.