Improved Semantic Representation Learning by Multiple Clustering for Image-Based 3D Model Retrieval

Abstract

Under the heavy management on the increasing 3D models, the topic of image-based 3D model retrieval which organizes unlabeled 3D models based on abundant knowledge learned from labeled 2D images has drawn attention. However, prior methods are limited in aligning semantically at corresponding categories of two domains due to the lack of label information in the 3D domain. To this end, this paper proposes an improved semantic representation learning by multiple clustering approach, which improves the reliability of pseudo labels for 3D models, so as to achieve class-level semantic alignment. Specifically, this paper first extracts features for 2D images and 3D models. Then it clusters combining the 3D features with the semantic information from multiple clustering on 3D model features to obtain more reliable target pseudo labels. Extensive experiments have shown that the proposed method has achieved the gain of 3.0%-205.0% averagely for popular retrieval metrics on the benchmark of monocular image-based 3D object retrieval (MI3DOR), and 1.3%-69.7% on another advanced benchmark, MI3DOR-2.