Convolutional networks for real-time 6-DOF camera relocalization.

Abstract

Figure 1: Convolutional neural network monocular camera relocalization. Relocalization results for an input image (top), the predicted camera pose of a visual reconstruction (middle), shown again overlaid in red on the original image (bottom). Our system relocalizes to within approximately 2m and 3 ◦ for large outdoor scenes spanning 50, 000m2. We present a robust and real-time monocular six de-gree of freedom relocalization system. Our system trains a convolutional neural network to regress the 6-DOF cam-era pose from a single RGB image in an end-to-end man-ner with no need of additional engineering or graph op-timisation. The algorithm can operate indoors and out-doors in real time, taking 5ms per frame to compute. It obtains approximately 2m and 3◦accuracy for large scale outdoor scenes and 0.5m and 5◦accuracy indoors. This is achieved using an efficient 23 layer deep convnet, demon-strating that convnets can be used to solve complicated out of image plane regression problems. This was made possi-ble by leveraging transfer learning from large scale classi-fication data. We show the convnet localizes from high level features and is robust to difficult lighting, motion blur and different camera intrinsics where point based SIFT registra-tion fails. Furthermore we show how the pose feature that is produced generalizes to other scenes allowing us to regress pose with only a few dozen training examples. 1.