VISUAL ANALYSIS OF SINGLE‐CASE TIME SERIES: EFFECTS OF VARIABILITY, SERIAL DEPENDENCE, AND MAGNITUDE OF INTERVENTION EFFECTS

TLDR

Visual analysis is the dominant method for single‑case time‑series data, yet the literature assumes analysts will be conservative judges. This study investigates whether prior research has adequately examined false‑alarm and miss rates and the impact of serial dependence on visual analysis. Thirty‑seven postgraduate students evaluated 27 AB charts generated with a first‑order autoregressive model, varying effect size, random variability, and autocorrelation (0, 0.3, 0.6) in a factorial design. False‑alarm rates were unexpectedly high (16–84 %) and increased with positive autocorrelation and random variation, while miss rates were low (0–22 %) and largely unaffected, indicating analysts are not conservative and serial dependence influences judgment.

Abstract

Visual analysis is the dominant method of analysis for single-case time series. The literature assumes that visual analysts will be conservative judges. We show that previous research into visual analysis has not adequately examined false alarm and miss rates or the effect of serial dependence. In order to measure false alarm and miss rates while varying serial dependence, amount of random variability, and effect size, 37 students undertaking a postgraduate course in single-case design and analysis were required to assess the presence of an intervention effect in each of 27 AB charts constructed using a first-order autoregressive model. Three levels of effect size and three levels of variability, representative of values found in published charts, were combined with autocorrelation coefficients of 0, 0.3 and 0.6 in a factorial design. False alarm rates were surprisingly high (16% to 84%). Positive autocorrelation and increased random variation both significantly increased the false alarm rates and interacted in a nonlinear fashion. Miss rates were relatively low (0% to 22%) and were not significantly affected by the design parameters. Thus, visual analysts were not conservative, and serial dependence did influence judgment.