The Clinical Teacher · 2014 · 25 citations · 8 references
EducationEducational AssessmentEducational MeasurementPsychometricsStandard-settingStandardized TestingBorderline Group MethodCriterion-referenced Standard SettingSkills AssessmentStandard-setting ReliabilityStandard-setting ValidityStandardized ExaminationsAssessment Rating ScalesBorderline CasesChecklist ScoringClinical Rating ScalesGlobal Rating ScalesPassing Score DeterminationPassing Score SettingPassing Scores
BACKGROUND: The evaluation of clinical skills and competencies is a high-stakes process carrying significant consequences for the candidate. Hence, it is mandatory to have a robust method to justify the pass score in order to maintain a valid and reliable objective structured clinical examination (OSCE). The aim was to trial the borderline approach using the two-domain global rating scale for standard setting in the OSCE. METHODS: For each domain, a set of six-point (from 5 to 0) scales were used to reflect high and low divisions within the 'pass', 'borderline' and 'fail' categories. Scores on the two individual global scales were summed to create a 'summed global rating'. Similarly task-based checklists for individual stations were summed to get a total score. It is mandatory to have a robust method to justify the pass score in order to maintain a valid and reliable OSCE RESULTS: The Pearson's correlation between task-based checklist scoring and the two-domain global rating scale were moderate and significant. The highest R(2) coefficient of 0.479 was obtained for station 7, and the lowest R(2) value was 0.241 for station 14. DISCUSSION: There was a significant positive correlation between the two scales; however, the R(2) value was not satisfactory except for station 7. The pass mark for the OSCE according to the borderline method was 64 per cent, which is higher than the arbitrarily set pass mark of 50 per cent. CONCLUSIONS: This study confirms that the two-domain global rating scale is appropriate to assess the abilities of students within the framework of an OSCE. The strong relationships between the two-domain global rating scale and task-based checklists provide evidence that the two-domain global rating scale can be used to genuinely assess students' proficiencies.
8
Setting standards on educational tests
John J. Norcini · Medical Education · 2003 · 314 citations
Education, Educational Assessment, Educational Measurement +13
Guidelines for assessing clinical competence
David Newble, Beth Dawson, Dale Dauphinée et al. · Teaching and Learning in Medicine · 1994 · 102 citations · Full text
Standards and reliability in evaluation
John J. Norcini · Academic Medicine · 1999 · 58 citations
Education, Educational Assessment, Educational Measurement +14