Cognitive Assessment Models with Few Assumptions, and Connections with Nonparametric Item Response Theory

TLDR

The paper examines single‑strategy cognitive assessment models that assume a stochastic conjunctive relationship between latent attributes and task performance, highlighting usability and interpretability issues while noting that these models make few assumptions beyond the conjunctive structure. The authors explore stochastic ordering and monotonicity properties to improve model interpretability. An example demonstrates that the models can be sensitive to cognitive attributes even when data fit the Rasch model, and simple data summaries are identified to detect the presence or absence of attributes when full estimation is infeasible.

Abstract

Some usability and interpretability issues for single-strategy cognitive assessment models are considered. These models posit a stochastic conjunctive relationship between a set of cognitive attributes to be assessed and performance on particular items/tasks in the assessment. The models considered make few assumptions about the relationship between latent attributes and task performance beyond a simple conjunctive structure. An example shows that these models can be sensitive to cognitive attributes, even in data designed to well fit the Rasch model. Several stochastic ordering and monotonicity properties are considered that enhance the interpretability of the models. Simple data summaries are identified that inform about the presence or absence of cognitive attributes when the full computational power needed to estimate the models is not available.