Evaluating the usability of an evolving collaborative product —changes in user type, tasks and evaluation methods over time

Abstract

The first users of a new technology are often engineers and enthusiasts. The functionality and interface that they find acceptable may be very different than the requirements of a more mainstream audience. This poses challenges for usability engineers in both defining user groups and then evaluating a product against usability goals, when both users and goals are changing as the technology matures. Usability evaluation methods for collaborative applications must evolve and iterate at least as fast as the products themselves. This paper describes the changes in approach taken by usability engineers between Version 1 and Version 3 of the Microsoft NetMeeting product.