Keyframe-based visual–inertial odometry using nonlinear optimization

TLDR

Visual–inertial odometry combines complementary visual and inertial sensors, and recent advances in nonlinear optimization promise higher accuracy than traditional filtering while remaining tractable. The study aims to develop a probabilistic cost function that fuses landmark reprojection errors with inertial terms for visual–inertial odometry. They implement a sliding‑window nonlinear optimization that fuses landmark reprojection and inertial terms, marginalizing past keyframes to keep the problem tractable and enabling real‑time operation on custom stereo visual–inertial hardware. Evaluation on complementary stereo and monocular datasets shows that the proposed method outperforms both a baseline stereo/monocular implementation and a state‑of‑the‑art stochastic cloning filter, achieving higher accuracy at the cost of increased computation.

Abstract

Combining visual and inertial measurements has become popular in mobile robotics, since the two sensing modalities offer complementary characteristics that make them the ideal choice for accurate visual–inertial odometry or simultaneous localization and mapping (SLAM). While historically the problem has been addressed with filtering, advancements in visual estimation suggest that nonlinear optimization offers superior accuracy, while still tractable in complexity thanks to the sparsity of the underlying problem. Taking inspiration from these findings, we formulate a rigorously probabilistic cost function that combines reprojection errors of landmarks and inertial terms. The problem is kept tractable and thus ensuring real-time operation by limiting the optimization to a bounded window of keyframes through marginalization. Keyframes may be spaced in time by arbitrary intervals, while still related by linearized inertial terms. We present evaluation results on complementary datasets recorded with our custom-built stereo visual–inertial hardware that accurately synchronizes accelerometer and gyroscope measurements with imagery. A comparison of both a stereo and monocular version of our algorithm with and without online extrinsics estimation is shown with respect to ground truth. Furthermore, we compare the performance to an implementation of a state-of-the-art stochastic cloning sliding-window filter. This competitive reference implementation performs tightly coupled filtering-based visual–inertial odometry. While our approach declaredly demands more computation, we show its superior performance in terms of accuracy.