An Insight-Based Methodology for Evaluating Bioinformatics Visualizations

TLDR

High‑throughput experiments generate large datasets, prompting the development of many visualization tools aimed at delivering biologically relevant insights. The study aims to evaluate and rank bioinformatics visualizations by developing a real‑world, insight‑focused evaluation method. The authors assessed five microarray visualization tools by measuring the quantity, types, and acquisition time of insight based on characteristics derived from real‑world data analysis scenarios. The study identified insight characteristics that can be quantified in open‑ended tests, showing that tool choice depends on data type, interaction techniques influence insight, and the method offers a new evaluation approach applicable beyond bioinformatics.

Abstract

High-throughput experiments, such as gene expression microarrays in the life sciences, result in very large data sets. In response, a wide variety of visualization tools have been created to facilitate data analysis. A primary purpose of these tools is to provide biologically relevant insight into the data. Typically, visualizations are evaluated in controlled studies that measure user performance on predetermined tasks or using heuristics and expert reviews. To evaluate and rank bioinformatics visualizations based on real-world data analysis scenarios, we developed a more relevant evaluation method that focuses on data insight. This paper presents several characteristics of insight that enabled us to recognize and quantify it in open-ended user tests. Using these characteristics, we evaluated five microarray visualization tools on the amount and types of insight they provide and the time it takes to acquire it. The results of the study guide biologists in selecting a visualization tool based on the type of their microarray data, visualization designers on the key role of user interaction techniques, and evaluators on a new approach for evaluating the effectiveness of visualizations for providing insight. Though we used the method to analyze bioinformatics visualizations, it can be applied to other domains.