The use of visual and auditory feedback for assembly task performance in a virtual environment

Abstract

This paper presents our creation and evaluation of multi-modal interface for a virtual assembly environment (VAE). It involves implementing an assembly simulation environment with multi-sensory feedback (visual and auditory), and evaluating the effects of multi-modal feedback on assembly task performance. This virtual environment experimental platform brought together complex technologies such as constraint-based assembly simulation. optical motion tracking technology, and real-time 3D sound generation technology around a virtual reality (VR) workbench and a common software platform. Peg-in-a-hole and Sener electronic box assembly tasks have been used as the task cases to perform human factor experiments, using sixteen subjects. Both objective performance data (task completion time, and human performance error rates) and subjective opinions (questionnaires) have been gathered from these experiments. The results showed that the addition of 3D auditory or visual feedback did introduce an improvement in the virtual assembly task performance. They also indicated that the integrated feedback (visual plus auditory) offered better task performance than any feedback used in isolation. Most of the users preferred the combined feedback to any individual feedback (visual or auditory) or no feedback.