Iterated extended Kalman filter based visual-inertial odometry using direct photometric feedback

TLDR

The authors propose a visual‑inertial odometry system that tightly fuses camera and IMU data using an iterated extended Kalman filter. The system uses image patches as photometric landmarks, integrates their error directly into the filter, operates in a fully robocentric state, and allows undelayed landmark initialization without separate feature extraction. Experiments demonstrate that the approach tracks non‑corner features, remains robust to low texture and motion blur, provides pose estimates from the second frame, and achieves high‑accuracy localization comparable to state‑of‑the‑art methods.

Abstract

This paper presents a visual-inertial odometry framework that tightly fuses inertial measurements with visual data from one or more cameras, by means of an iterated extended Kalman filter. By employing image patches as landmark descriptors, a photometric error is derived, which is directly integrated as an innovation term in the filter update step. Consequently, the data association is an inherent part of the estimation process and no additional feature extraction or matching processes are required. Furthermore, it enables the tracking of noncorner-shaped features, such as lines, and thereby increases the set of possible landmarks. The filter state is formulated in a fully robocentric fashion, which reduces errors related to nonlinearities. This also includes partitioning of a landmark’s location estimate into a bearing vector and distance and thereby allows an undelayed initialization of landmarks. Overall, this results in a compact approach, which exhibits a high level of robustness with respect to low scene texture and motion blur. Furthermore, there is no time-consuming initialization procedure and pose estimates are available starting at the second image frame. We test the filter on different real datasets and compare it with other state-of-the-art visual-inertial frameworks. Experimental results show that robust localization with high accuracy can be achieved with this filter-based framework.