Toward measuring visualization insight

TLDR

Visualization research has increasingly focused on evaluation, prompting a reexamination of its ultimate purpose, which some argue is to generate insight. The study aims to assess how well visualizations generate insight, enabling direct comparison of design alternatives against an insight goal. The authors evaluate the controlled experiment method as a means to measure insight.

Abstract

Recent visualization research literature has paid an increasing amount of attention to evaluating visualizations. It seems an appropriate time to reopen the question about what the ultimate purpose of visualization is and how it should be evaluated. One potential claim is: the purpose of visualization is insight. The purpose of visualization evaluation is to determine to what degree visualizations achieve this purpose. If this claim is true, then evaluating visualizations should seek to determine how well visualizations generate insight. Measuring insight would enable the direct comparison of visualization design alternatives, or the comparison against an insight goal. This article examines the capability of the controlled experiment method to measure insight