MOTION AND STRUCTURE FROM MOTION IN A PIECEWISE PLANAR ENVIRONMENT

TLDR

For points and lines on the same plane, the correspondence between two cameras is a collineation. The study demonstrates that a piecewise linear environment constrains image matches under unknown camera motion and applies this insight to calibrate camera systems. Ambiguity is resolved by using a second plane, a third view, or prior plane geometry, and the authors combine collineation matrix estimation with point and line matches through hypothesis prediction and testing guided by a Kalman filter. The authors find that a piecewise linear environment constrains image matches, enabling recovery of camera motion and plane equations from the collineation matrix, and that combining this with hypothesis testing guided by a Kalman filter allows accurate camera system calibration.

Abstract

We show in this article that when the environment is piecewise linear, it provides a powerful constraint on the kind of matches that exist between two images of the scene when the camera motion is unknown. For points and lines located in the same plane, the correspondence between the two cameras is a collineation. We show that the unknowns (the camera motion and the plane equation) can be recovered, in general, from an estimate of the matrix of this collineation. The two-fold ambiguity that remains can be removed by looking at a second plane, by taking a third view of the same plane, or by using a priori knowledge about the geometry of the plane being looked at. We then show how to combine the estimation of the matrix of collineation and the obtaining of point and line matches between the two images, by a strategy of Hypothesis Prediction and Testing guided by a Kalman filter. We finally show how our approach can be used to calibrate a system of cameras.