Unified temporal and spatial calibration for multi-sensor systems

TLDR

Robotics state estimation increasingly relies on multiple complementary sensors, which must be accurately spatially and temporally registered for optimal sensor fusion. This paper proposes a novel framework that jointly estimates temporal offsets and spatial displacements between sensors. The method employs continuous‑time batch estimation under a maximum‑likelihood framework, seamlessly incorporating time offsets. Experiments calibrating a camera and IMU demonstrate accurate estimation of time offsets down to a fraction of the smallest measurement period.

Abstract

In order to increase accuracy and robustness in state estimation for robotics, a growing number of applications rely on data from multiple complementary sensors. For the best performance in sensor fusion, these different sensors must be spatially and temporally registered with respect to each other. To this end, a number of approaches have been developed to estimate these system parameters in a two stage process, first estimating the time offset and subsequently solving for the spatial transformation between sensors. In this work, we present on a novel framework for jointly estimating the temporal offset between measurements of different sensors and their spatial displacements with respect to each other. The approach is enabled by continuous-time batch estimation and extends previous work by seamlessly incorporating time offsets within the rigorous theoretical framework of maximum likelihood estimation. Experimental results for a camera to inertial measurement unit (IMU) calibration prove the ability of this framework to accurately estimate time offsets up to a fraction of the smallest measurement period.