Evaluation of Stereo Matching Costs on Images with Radiometric Differences

TLDR

Stereo correspondence methods depend on matching costs to assess image similarity. The study evaluates how insensitive various stereo matching costs are to radiometric variations in passive binocular stereo. The authors compared a range of pixel‑, window‑, filter‑, and nonparametric matching costs—including AD, SAD, NCC, BilSub, Rank, Census, and HMI—across local, semi‑global, and global stereo algorithms on datasets with controlled exposure, lighting, vignetting, and noise variations. Across all datasets and experiments, the cost ranking was consistent, with BilSub excelling for low radiometric differences, HMI performing best under strong noise, and Census delivering the most robust overall performance.

Abstract

Stereo correspondence methods rely on matching costs for computing the similarity of image locations. We evaluate the insensitivity of different costs for passive binocular stereo methods with respect to radiometric variations of the input images. We consider both pixel-based and window-based variants like the absolute difference, the sampling-insensitive absolute difference, and normalized cross correlation, as well as their zero-mean versions. We also consider filters like LoG, mean, and bilateral background subtraction (BilSub) and nonparametric measures like Rank, SoftRank, Census, and Ordinal. Finally, hierarchical mutual information (HMI) is considered as pixelwise cost. Using stereo data sets with ground-truth disparities taken under controlled changes of exposure and lighting, we evaluate the costs with a local, a semiglobal, and a global stereo method. We measure the performance of all costs in the presence of simulated and real radiometric differences, including exposure differences, vignetting, varying lighting, and noise. Overall, the ranking of methods across all data sets and experiments appears to be consistent. Among the best costs are BilSub, which performs consistently very well for low radiometric differences; HMI, which is slightly better as pixelwise matching cost in some cases and for strong image noise; and Census, which showed the best and most robust overall performance.