Parameter Recovery in the Graded Response Model Using MULTILOG

TLDR

The graded response model describes test‑taking behavior when item responses are classified into ordered categories. The study investigated parameter recovery in the graded response model using the MULTILOG computer program under default conditions. Thirty‑six simulated data sets with five response categories were generated, varying true θ distribution, item discrimination distribution, and calibration sample size. The study found that with calibration samples of at least 500 examinees, true and estimated parameters correlated above 0.85, root‑mean‑square errors were comparable to binary data studies, and while small samples mainly affected item‑parameter recovery, person‑parameter estimates remained robust, indicating that 500 examinees are required for adequate calibration.

Abstract

The graded response model can be used to describe test‐taking behavior when item responses are classified into ordered categories. In this study, parameter recovery in the graded response model was investigated using the MULTILOG computer program under default conditions. Based on items having five response categories, 36 simulated data sets were generated that varied on true θ distribution, true item discrimination distribution, and calibration sample size. The findings suggest, first, the correlations between the true and estimated parameters were consistently greater than 0.85 with sample sizes of at least 500. Second, the root mean square error differences between true and estimated parameters were comparable with results from binary data parameter recovery studies. Of special note was the finding that the calibration sample size had little influence on the recovery of the true ability parameter but did influence item‐parameter recovery. Therefore, it appeared that item‐parameter estimation error, due to small calibration samples, did not result in poor person‐parameter estimation. It was concluded that at least 500 examinees are needed to achieve an adequate calibration under the graded model.