Predictors of Rater Bias in the Assessment of Social-Emotional Competence

Abstract

The Devereux Student Strengths Assessment Mini (DESSA-Mini) (LeBuffe, Shapiro, &
\nNaglieri, 2014) efficiently monitors the growth of Social-Emotional Competence (SEC)
\nin the routine implementation of Social Emotional Learning programs. The DESSAMini
\nis used to assess approximately half a million children around the world. Since
\nbehavior rating scales can have ‘rater bias’, this paper examines rater characteristics that
\ncontribute to DESSA-Mini ratings. Rater characteristics and DESSA-Mini ratings were
\ncollected from elementary school classroom teachers (n=72) implementing TOOLBOX
\nin a racially/ethnically diverse California school district. Teachers rated 1,676 students,
\nwho scored similarly to a national reference group. Multilevel modeling analysis showed
\nthat only 16% of variance in DESSA-mini ratings was attributable to raters.
\nRelationships between teacher characteristics and ratings were estimated to examine
\nrater variance. Collectively, four characteristics of teachers (perceived barriers to student
\nlearning, sense of their ‘typical’ student’s level of SEC, anticipation of SEL program
\nimplementation challenges, and intentions to fully implement a newly adopted SEL
\nprogram) accounted for bias in teacher-generated DESSA scores, leaving only 10% of
\nthe variance unexplained. Identified sources of ‘rater bias’ can be controlled for in
\nresearch and addressed through thoughtful program selection, training, and
\nimplementation.