Event analysis and interpretation of human activity for augmented reality-based assistant systems

Abstract

In this paper a concept and its implementation of an ergonomic cognitive assistant system for supporting human workers at complex assembly tasks in industrial environments is introduced. Depending on the level of the userpsilas product knowledge this mixed-initiative system follows and gains knowledge from the human workerpsilas construction steps while it is also able to automatically give hints and instruct the worker whenever needed. The presented agent bases on a closed human-machine interaction loop consisting of the multimodal perception of the workerpsilas action, the comparison with the systempsilas knowledge about the production task, and the displaying of the adequate next assembly instruction step. First experimental results of the assistant system are demonstrated on a simplified use case with the construction of a small toy car using augmented reality display techniques.