Real-Time Video-Based Reconstruction of Urban Environments

Abstract

We present an approach for automatic 3D reconstruction of outdoor scenes using computer vision techniques. Our system collectsvideo, GPS and INS data which are processed in real-time to produce geo-registered, detailed 3D models that represent the geometryand appearance of the world. These models are generated without manual measurements or markers in the scene and can be used forvisualization from arbitrary viewpoints, documentation and archiving of large areas. Our system consists of a data acquisition systemand a processing system that generated 3D models from the video sequences off-line but in real-time. The GPS/INS measurementsallow us to geo-register the pose of the camera at the time each frame was captured. The following stages of the processing pipelineperform dense reconstruction and generate the 3D models, which are in the form of a triangular mesh and a set of images that providetexture. By leveraging the processing power of the GPU, we are able to achieve faster than real-time performance, while maintainingan accuracy of a few cm.