Fusing points and lines for high performance tracking

TLDR

The paper tackles real‑time 3D model‑based tracking by fusing point‑ and edge‑based systems, integrating their pose estimates, applying online learning to improve feature tracking, and deploying the FAST detector for 400 Hz full‑frame feature extraction. The authors combine the two trackers by fusing pose estimates, use online learning to refine feature tracking, and employ the FAST detector to enable real‑time performance. The resulting system achieves extremely high performance, with average prediction errors of 200 pixels and the capability to track rapid motions up to 50 deg/s at 6 Hz.

Abstract

This paper addresses the problem of real-time 3D model-based tracking by combining point-based and edge-based tracking systems. We present a careful analysis of the properties of these two sensor systems and show that this leads to some non -trivial design choices that collectively yield extremely high performance. In particular, we present a method for integrating the two systems and robustly combining the pose estimates they produce. Further we show how on-line learning can be used to improve the performance of feature tracking. Finally, to aid real-time performance, we introduce the FAST feature detector which can perform full-frame feature detection at 400Hz. The combination of these techniques results in a system which is capable of tracking average prediction errors of 200 pixels. This level of robustness allows us to track very rapid motions, such as 50deg camera shake at 6Hz