Pose tracking from natural features on mobile phones

TLDR

SIFT is a strong but computationally expensive descriptor, while Ferns is fast but memory‑intensive, making both unsuitable for mobile phones. The paper presents two real‑time natural feature tracking techniques for mobile phones. We heavily modify SIFT and Ferns descriptors to enable real‑time tracking on mobile devices. The techniques achieve up to 20 Hz on current‑generation phones, with evaluations showing robust performance suitable for augmented reality.

Abstract

In this paper we present two techniques for natural feature tracking in real-time on mobile phones. We achieve interactive frame rates of up to 20 Hz for natural feature tracking from textured planar targets on current-generation phones. We use an approach based on heavily modified state-of-the-art feature descriptors, namely SIFT and Ferns. While SIFT is known to be a strong, but computationally expensive feature descriptor, Ferns classification is fast, but requires large amounts of memory. This renders both original designs unsuitable for mobile phones. We give detailed descriptions on how we modified both approaches to make them suitable for mobile phones. We present evaluations on robustness and performance on various devices and finally discuss their appropriateness for augmented reality applications.