Interrater Agreement of the Individualized Behavior Rating Scale Tool

Abstract

Data assessment is critical for determining student behavior change in response to individualized behavior interventions in schools. This study examined the interrater agreement of the Individualized Behavior Rating Scale Tool (IBRST), a perceptual direct behavior rating tool that was used by typical school personnel to record behavior occurrence in students requiring individualized interventions. Two independent observers (teacher and data collector) used the IBRST to rate student-specific problem and appropriate behaviors during specified observation times. Data were collected across 19 students and agreement between raters was compared. Resulting linear- and quadratic-weighted kappa coefficients indicated generally adequate agreement between raters on problem behaviors and appropriate behaviors. When ratings were categorized into more or less salient behaviors, less than adequate agreement (<.60) was found for some behaviors that were less salient. Agreement remained stable from baseline to intervention. Implications for practice, limitations of the study, and directions for future research are discussed.