3-D Motion and Structure Estimation Using Inertial Sensors and Computer Vision for Augmented Reality

Abstract

A new method for registration in augmented reality (AR) was developed that simultaneously tracks the position, orientation, and motion of the user's head, as well as estimating the three-dimensional (3-D) structure of the scene. The method fuses data from headmounted cameras and head-mounted inertial sensors. Two Extended Kalman Filters (EKF) are used; one of which estimates the motion of the user's head and the other that estimates the 3-D locations of points in the scene. A recursive loop is used between the two EKFs. The algorithm was tested using a combination of synthetic and real data, and in general was found to perform well. A further test showed that a system using two cameras performed much better than a system using a single camera, although improving the accuracy of the inertial sensors can partially compensate for the loss of one camera. The method is suitable for use in completely unstructured and unprepared environments. Unlike previous work in this area, this method requires no a priori knowledge about the scene, and can work in environments where the objects of interest are close to the user. Index terms: Augmented reality, pose estimation, registration, Kalman filter, structure from motion, computer vision, inertial sensors Submitted to Presence: Teleoperators and Virtual Environments 3 1