Regional Maximum Activations of Convolutions with Attention for Cross-domain Beauty and Personal Care Product Retrieval

Abstract

Cross-domain beauty and personal care product image retrieval is a challenging problem due to data variations (e.g., brightness, viewpoint, and scale), and the rich types of items. In this paper, we present a regional maximum activations of convolutions with attention (RA-MAC) descriptor to extract image features for retrieval. RA-MAC improves the regional maximum activations of convolutions (R-MAC) descriptor considering the influence of background in cross-domain images (i.e., shopper domain and seller domain). More specifically, RA-MAC utilizes the characteristics of the convolutional layer to find the attention of an image, and reduces the influence of the unimportant regions in an unsupervised manner. Furthermore, a few strategies have been exploited to improve the performance, such as multiple features fusion, query expansion, and database augmentation. Extensive experiments conducted on a dataset consisting of half a million images of beauty care products (Perfect-500K) manifest the effectiveness of RA-MAC. Our approach achieves the 2nd place in the leader board of the Grand Challenge of AI Meets Beauty in ACM Multimedia 2018. Our code is available at: https://github.com/RetrainIt/Perfect-Half-Million-Beauty-Product-Image-Recognition-Challenge.