Classification of Cultural Heritage Sites Using Transfer Learning

Abstract

India is a country endowed with rich cultural heritage especially renowned architectural sites of which 37 are UNESCO listed heritage sites. Cultural heritages connects generations over time and we need to preserve them. Architects, historians, travelers etc. They visit many historical sites where it often becomes difficult for them to identify and get historical details about the monument they are interested in. The task of archiving, documenting and sharing the knowledge of these cultural assets is challenging due to the scale and reliability of the information. An accurate prediction of the images to its correct label (heritage site) allows more proficient searches through specific terms, thus helping in the studying and understanding the heritage assets. Classification of data, which involve images, is complex and also time consuming. The present state of art of machine learning techniques can be harnessed to atomize the classification of images. The advent of high processing computing resources, state of art machine learning and deep learning algorithms provide tangible solutions. In this paper, we propose a crowdsourcing platform to collect Indian Digital Heritage Space (IHDS) monuments data, perform image classification and query based retrieval of Image labels. Further we designed a transfer learning based image classifier, which retrained using IHDS dataset on pre-trained MobileNet V2 architecture over ImageNet Dataset. The proposed transfer learning algorithm shows inference accuracy of 98.75%. We demonstrate our crowd source framework using a web application, which is part of Indian Digital Heritage Space (IHDS) project funded by DST, Government of India, New Delhi.