SLAM++: Simultaneous Localisation and Mapping at the Level of Objects

TLDR

The authors introduce an object‑oriented 3D SLAM paradigm that exploits the fact that many scenes contain repeated, domain‑specific objects and structures. Using a hand‑held depth camera, the system performs real‑time 3D object recognition and tracking to produce 6DoF camera‑object constraints that populate an explicit object graph refined by pose‑graph optimisation, enabling accurate ICP‑based camera‑to‑model tracking and active search for new objects. The method delivers real‑time incremental SLAM in large, cluttered environments with loop closure, relocalisation, moved‑object detection, and generates compressed object‑level scene descriptions, providing the predictive power of dense SLAM with far less representation.

Abstract

We present the major advantages of a new 'object oriented' 3D SLAM paradigm, which takes full advantage in the loop of prior knowledge that many scenes consist of repeated, domain-specific objects and structures. As a hand-held depth camera browses a cluttered scene, real-time 3D object recognition and tracking provides 6DoF camera-object constraints which feed into an explicit graph of objects, continually refined by efficient pose-graph optimisation. This offers the descriptive and predictive power of SLAM systems which perform dense surface reconstruction, but with a huge representation compression. The object graph enables predictions for accurate ICP-based camera to model tracking at each live frame, and efficient active search for new objects in currently undescribed image regions. We demonstrate real-time incremental SLAM in large, cluttered environments, including loop closure, relocalisation and the detection of moved objects, and of course the generation of an object level scene description with the potential to enable interaction.