Robust Wide Baseline Stereo from Maximally Stable Extremal Regions

TLDR

Wide‑baseline stereo seeks correspondences between images from different viewpoints, a task challenged by large scale, illumination, rotation, occlusion, anisotropic scaling, and viewpoint translation. The study introduces extremal regions and a robust similarity measure to establish tentative correspondences in wide‑baseline stereo. Extremal regions are invariant to projective and intensity changes; an efficient near‑linear algorithm detects maximally stable extremal regions (MSER), and invariants from larger measurement regions are used to compute robust correspondences. Experiments on indoor and outdoor image pairs show that MSERs, measurement regions, and the robust metric yield accurate epipolar geometry with average point‑to‑line error below 0.09 pixel.

Abstract

Abstract The wide-baseline stereo problem, i.e. the problem of establishing correspondences between a pair of images taken from different viewpoints is studied. A new set of image elements that are put into correspondence, the so called extremal regions , is introduced. Extremal regions possess highly desirable properties: the set is closed under (1) continuous (and thus projective) transformation of image coordinates and (2) monotonic transformation of image intensities. An efficient (near linear complexity) and practically fast detection algorithm (near frame rate) is presented for an affinely invariant stable subset of extremal regions, the maximally stable extremal regions (MSER). A new robust similarity measure for establishing tentative correspondences is proposed. The robustness ensures that invariants from multiple measurement regions (regions obtained by invariant constructions from extremal regions), some that are significantly larger (and hence discriminative) than the MSERs, may be used to establish tentative correspondences. The high utility of MSERs, multiple measurement regions and the robust metric is demonstrated in wide-baseline experiments on image pairs from both indoor and outdoor scenes. Significant change of scale (3.5×), illumination conditions, out-of-plane rotation, occlusion, locally anisotropic scale change and 3D translation of the viewpoint are all present in the test problems. Good estimates of epipolar geometry (average distance from corresponding points to the epipolar line below 0.09 of the inter-pixel distance) are obtained.