Semantic Modeling of Places using Objects

Abstract

While robot mapping has seen massive strides recently, higher level abstractions in map representation are still not widespread. Maps containing semantic concepts such as objects and labels are essential for many tasks in manmade environments as well as for human-robot interaction and map communication. In keeping with this aim, we present a model for places using objects as the basic unit of representation. Our model is a 3D extension of the constellation object model, popular in computer vision, in which the objects are modeled by their appearance and shape. The 3D location of each object is maintained in a coordinate frame local to the place. The individual object models are learned in a supervised manner using roughly segmented and labeled training images. Stereo range data is used to compute 3D locations of the objects. We use the Swendsen-Wang algorithm, a cluster MCMC method, to solve the correspondence problem between image features and objects during inference. We provide a technique for building panoramic place models from multiple views of a location. An algorithm for place recognition by comparing models is also provided. Results are presented in the form of place models inferred in an indoor environment. We envision the use of our place model as a building block towards a complete object-based semantic mapping system.