A Tutorial on Quantitative Trajectory Evaluation for Visual(-Inertial) Odometry

TLDR

This tutorial presents principled methods for quantitatively evaluating visual‑inertial odometry trajectories, establishing a benchmark foundation and generalizing the approach to any sensing modality. The authors detail how to select alignment transformations per modality, define absolute and relative error metrics, extend the formulation to arbitrary sensing setups, and provide a publicly available implementation to support reproducibility.

Abstract

In this tutorial, we provide principled methods to quantitatively evaluate the quality of an estimated trajectory from visual(-inertial) odometry (VO/VIO), which is the foundation of benchmarking the accuracy of different algorithms. First, we show how to determine the transformation type to use in trajectory alignment based on the specific sensing modality (i.e., monocular, stereo and visual-inertial). Second, we describe commonly used error metrics (i.e., the absolute trajectory error and the relative error) and their strengths and weaknesses. To make the methodology presented for VO/VIO applicable to other setups, we also generalize our formulation to any given sensing modality. To facilitate the reproducibility of related research, we publicly release our implementation of the methods described in this tutorial.