Combining Data on Criticality and Frequency in Developing Test Plans for Licensure and Certification Examinations

Abstract

Job analysis is a critical component in evaluating the validity of many high‐stakes testing programs, particularly those used for licensure or certification. The ratings of criticality and frequency of various activities that are derived from such job analyses can be combined in a number of ways. This paper develops a multiplicative model as a natural and effective way to combine ratings o f frequency and criticality in order to obtain estimates of the relative importance of different activities for practice. An example of the model's use is presented. The multiplicative model incorporates adjustments to ensure that the effective weights of frequency and criticality are appropriate.