ESTIMATION OF LATENT ABILITY USING A RESPONSE PATTERN OF GRADED SCORES1

TLDR

The paper discusses estimating latent ability from full response patterns of free‑response items, extending to graded scoring and homogeneous thinking processes, and examines how item characteristic functions relate to scoring philosophy. The authors employ maximum likelihood, Bayes modal, and a Bayes estimator based on mean‑square error weighted by the latent density, and compare these estimators to conventional dichotomous or test‑score methods in terms of information, standard errors, and mean‑square errors. They establish sufficient conditions for unique maximum likelihood and Bayes modal estimators at the item level, demonstrate that these estimators outperform conventional methods in information and error metrics, and show that the Bayes modal estimator offers a computationally efficient compromise.

Abstract

ABSTRACT Estimation of latent ability using the entire response pattern of free‐response items is discussed, first in the general case and then in the case where the items are scored in a graded way, especially when the thinking process required for solving each item is assumed to be homogeneous. The maximum likelihood estimator, the Bayes modal estimator, and the Bayes estimator obtained by using the mean‐square error multiplied by the density function of the latent variate as the loss function are taken as our estimators. Sufficient conditions for the existence of a unique maximum likelihood estimator and a unique Bayes modal estimator are formulated with respect to an individual item rather than with respect to a whole set of items, which are useful especially in the situation where we are free to choose optimal items for a particular examinee out of the item library in which a sufficient number of items are stored with reliable quality controls. Advantages of the present methods are investigated by comparing them with those which make use of conventional dichotomous items or test scores, theoretically as well as empirically, in terms of the amounts of information, the standard errors of estimators, and the mean‐square errors of estimators. The utility of the Bayes modal estimator as a computational compromise for the Bayes estimator is also discussed and observed. The relationship between the formula for the item characteristic function and the philosophy of scoring is observed with respect to dichotomous items.