Exploration and Explanation in Computational Notebooks

TLDR

Computational notebooks integrate code, visualizations, and text, and are rapidly adopted by researchers, data analysts, and journalists. The study presents three investigations into how computational notebooks are used to document and share exploratory data analyses. The studies reveal that many notebooks lack explanatory text, that most academic notebooks describe methods but rarely reasoning or results, that analysts view notebooks as personal and messy, and that there is a tension between exploration and explanation, suggesting opportunities to promote explanation without hindering exploration.

Abstract

Computational notebooks combine code, visualizations, and text in a single document. Researchers, data analysts, and even journalists are rapidly adopting this new medium. We present three studies of how they are using notebooks to document and share exploratory data analyses. In the first, we analyzed over 1 million computational notebooks on GitHub, finding that one in four had no explanatory text but consisted entirely of visualizations or code. In a second study, we examined over 200 academic computational notebooks, finding that although the vast majority described methods, only a minority discussed reasoning or results. In a third study, we interviewed 15 academic data analysts, finding that most considered computational notebooks personal, exploratory, and messy. Importantly, they typically used other media to share analyses. These studies demonstrate a tension between exploration and explanation in constructing and sharing computational notebooks. We conclude with opportunities to encourage explanation in computational media without hindering exploration.