How Do Missing Data Bias Estimates of Within-Group Agreement? Sensitivity of <i>SD _WG, CV_WG, r_WG(J), r_WG(J)</i>* , and ICC to Systematic Nonresponse

Abstract

In multilevel theory testing, estimation of group-level properties (i.e., consensus and diversity) is often complicated by missing data. Researchers are left to draw inferences about group constructs (e.g., organizational climate and climate strength) from the responses of only a subset of group members. This study analyzes the biasing impact of random and non-random missingness patterns on within-group agreement and reliability (standard deviation, coefficient of variation, r WG(J) , r * WG(J) , AD M , a WG , and intraclass correlation) across a range of response rates, numbers of items, and systematic missing data mechanisms. Results demonstrate biases up to 20% over- or underestimation for common response rates found in organizational research. Correction formulae are presented, which enable assessment of the sensitivity of multilevel results to survey nonresponse.