Situated evaluation for cooperative systems

Abstract

This paper discusses an evaluation of the MEAD prototype, a multi-user interface generator tool particularly for use in the context of Air Traffic Control (ATC). The procedures we adopted took the form of opportunistic and informal evaluation sessions with small user groups, including Air Traffic Controllers (ATCOs). We argue that informal procedures are a powerful and cost effective method for dealing with specific evaluation issues in the context of CSCW but that wider issues are more problematic. Most notably, identifying the “validity” or otherwise of CSCW systems requires that the context of use be taken seriously, necessitating a fundamental re-appraisal of the concept of evaluation.