Large-scale Classification of Fine-Art Paintings

Abstract

In the past few years, the number of fine-art collections that are dig- itized and publicly available has been growing rapidly. With the availability of such large collections of digitized artworks comes the need to develop multime- dia systems to archive and retrieve this pool of data. Measuring the visual similar- ity between artistic items is an essential step for such multimedia systems, which can benefit more high-level multimedia tasks. In order to model this similarity between paintings, we should extract the appropriate visual features for paintings and find out the best approach to learn the similarity metric based on these fea- tures. We investigate a comprehensive list of visual features and metric learning approaches to learn an optimized similarity measure between paintings. We de- velop a machine that is able to make aesthetic-related semantic-level judgments, such as predicting a painting’s style, genre, and artist, as well as providing simi- larity measures optimized based on the knowledge available in the domain of art historical interpretation. Our experiments show the value of using this similarity measure for the aforementioned prediction tasks.