Setting defensible performance standards on OSCEs and standardized patient examinations

TLDR

Standardized patient assessments and OSCEs are increasingly used for high‑stakes certification, but reliable, equitable decisions require defensible cut‑scores, a topic that has received little research compared to multiple‑choice exams. This article offers guidance on establishing defensible standards for performance‑based assessments, particularly those involving standardized patients. The authors review and contrast several standard‑setting methods, outlining their respective strengths and weaknesses. They also propose future research directions and potential enhancements to existing performance‑based standard‑setting protocols.

Abstract

Recently, standardized patient assessments and objective structured clinical examinations have been used for high-stakes certification and licensure decisions. In these testing situations, it is important that the assessments are standardized, the scores are accurate and reliable, and the resulting decisions regarding competence ar equitable and defensible. For the decisions to be valid, justifiable standards, or cut-scores, must beset. Unfortunately, unlike the body of research specifically dedicated to multiple-choice examinations, relatively little research has been conducted on standard-setting methods appropriate for use with performance-based assessments. The purpose of this article is to provide the reader with some guidance on how to set defensible standards on performance assessments, especially those that utilize standardized patients in simulated medical encounters. Various methods are discussed and contrasted, highlighting the relevant strengths and weaknesses. In addition, based on the prevailing literature and research, ideas for future studies and potential augmentations to current performance-based standard setting protocols are advanced.