Integrated Crowdsourcing Framework Using Deep Learning for Digitalization of Indian Heritage Infrastructure

Abstract

Every culture in the world reflects its magnificence and significance through the heritage infrastructure it conceives in the course of its civilization. India is composed of diverse cultures which reflect grandeur in the architectural heritage across its territory. The necessity for digitizing and storing the information of the heritage of our country is challenging due to sheer scale of cultural data collection and the reliability of sources. These challenges can be overcome by harnessing the present state of art of technologies. Advancement of technology has impacted every area of our social life and the need to document, preserve the ancestral wisdom to pass it down to generations is of prime importance. In this paper, we propose our work on building a fully-fledged web framework using emerging technologies to aid the preservation of cultural heritage site and its related data and to build a Deep Neural Network (DNN) which classifies Heritage sites with the crowdsourced data. We propose an automated crowd-sourcing web application for data management and storage of images collected and a custom deep learning system for transfer learning with an extension for incremental learning. The framework also facilitates transfer learning to retrain the pre-trained DNN architecture for the crowd-sourced data for continuous improvement of the model and initiate back-end jobs for transfer learning. We also demonstrate the crowd-sourcing operations designed with academic hierarchy as reference and show its efficient data storage structure. We also display the extension of the framework as web application to edge devices to accelerate Indian heritage in digital space. Finally, we present the workflow for achieving 98.75% accuracy for a transfer learned model in the proposed framework with the crowd-sourced dataset.