User studies: why, how, and when?

Abstract

User studies offer a scientifically sound method to measure a visualization's performance. Reasons abound for pursuing user studies, particularly when evaluating the strengths and weaknesses of different visualization techniques. A good starting point in any study is the scientific or visual design question to be examined. This drives the process of experimental design. A poorly designed experiment will yield results of only limited value. Although a comprehensive discussion of experimental design is beyond the scope of the article, we offer suggestions and lessons learned. We also describe how we designed experiments to answer important questions from our own research.