Process Data in NAEP: Past, Present, and Future

Abstract

This article reviews how National Assessment of Educational Progress (NAEP) has come to collect and analyze data about cognitive and behavioral processes (process data) in the transition to digital assessment technologies over the past two decades. An ordered five-level structure is proposed for describing the uses of process data. The levels in this hierarchy range from ignoring the processes (i.e., counting only the outcomes), to incorporating process data as auxiliary or essential in addition to the outcome, to modeling the process as the outcome itself, either holistically in a rubric score or in a measurement model that accounts for sequential dependencies. Historical examples of these different uses are described as well as recent results using nontraditional analytical approaches. In the final section, speculative future directions incorporating state-of-the-art technologies and analysis methods are described with an eye toward hard-to-measure constructs such as higher order problem-solving and collaboration.