A Survey of Usability Evaluation in Virtual Environments: Classification and Comparison of Methods

TLDR

Virtual environments (VEs) are a relatively new type of human–computer interface in which users perceive and act in a three‑dimensional world, and because designers cannot rely solely on traditional two‑dimensional interface guidelines, usability evaluation is crucial for VEs. This paper provides an overview of VE usability evaluation to organize and critically analyze diverse work from the field. The authors discuss issues that differentiate VE usability evaluation from traditional GUIs, review current VE evaluation methods, and propose a classification space that compares methods by user involvement, evaluation context, and result types. To illustrate the classification, the authors compare two existing evaluation approaches—testbed evaluation and sequential evaluation.

Abstract

Virtual environments (VEs) are a relatively new type of human–computer interface in which users perceive and act in a three-dimensional world. The designers of such systems cannot rely solely on design guidelines for traditional two-dimensional interfaces, so usability evaluation is crucial for VEs. This paper presents an overview of VE usability evaluation to organize and critically analyze diverse work from this field. First, we discuss some of the issues that differentiate VE usability evaluation from evaluation of traditional user interfaces such as GUIs. We also present a review of some VE evaluation methods currently in use, and discuss a simple classification space for VE usability evaluation methods. This classification space provides a structured means for comparing evaluation methods according to three key characteristics: involvement of representative users, context of evaluation, and types of results produced. Finally, to illustrate these concepts, we compare two existing evaluation approaches: testbed evaluation (Bowman, Johnson, & Hodges, 1999) and sequential evaluation (Gabbard, Hix, & Swan, 1999).