Setting standards on educational tests

TLDR

The paper introduces methods for setting standards in professional assessment. The authors describe how to set a standard score, outlining steps such as choosing the standard type, method, judges, meeting, cutpoint calculation, and post‑setting actions, and illustrate four common methods for written and clinical exams. The study identifies key criteria for selecting a standard‑setting method—alignment with test purpose, expert judgment, data, research, transparency, and due diligence—and emphasizes that credibility hinges on knowledgeable, representative setters and stakeholder acceptance, noting that standards reflect professional values and are established through systematic consensus.

Abstract

Objective This instalment in the series on professional assessment provides an introduction to methods of setting standards. Method A standard is a special score that serves as a boundary between those who perform well enough and those who do not. The practical steps in selecting it include: deciding on the type of standard; deciding the method for setting it; selecting the judges; holding the meeting; calculating the cutpoint, and deciding what to do afterwards. Four of the more popular methods are illustrated for both written and clinical examinations. Results The most important criteria for selecting a method for setting standards are whether it is consistent with the purpose of the test, based on expert judgement, informed by data, supported by research, transparent, and requires due diligence. The credibility of the standard will rely largely on the nature of the standard setters and the selection of a broadly representative and knowledgeable group is essential. After the standard has been set, it is important to ensure that stakeholders view the results as credible and that the pass rates have sensible relationships with other markers of competence. Conclusions A standard is an expression of professional values in the context of a test's purpose and content, the ability of the examinees, and the wider social or educational setting. Because standards are an expression of values, methods for setting them are systematic ways of gathering value judgements, reaching consensus and expressing that consensus as a single score on a test.