Indoor Augmented Reality Using Deep Learning for Industry 4.0 Smart Factories

Abstract

This paper proposes to design, develop and implement a fast and markerless mobile augmented reality system to achieve the registration for, the visualization of, and the interaction with machines in indoor smart factories with Industry 4.0 vision. A lightweight deep-learning image detection module based on MobileNets running on mobile devices is used to detect/recognize different machines and different portions of machines. Internet of Things (IoT) networking allows machines and sensors in machines to report data, such as machine settings and machine states, to the cloud-side server. Thus, augmented information associated with a machine portion can be derived from the server and superimposed with the portion image shown on the device display. Furthermore, interaction methods based on touch gestures and distance calculation are also implemented. A prototype system is developed and tested in a mechanical workshop for the purpose of validation and evaluation. The system is shown to achieve high detection accuracy, intuitive visualization, and unique interaction modes.