Evaluation & the Health Professions · 2007 · 81 citations · 17 references
EducationEducational AssessmentMedical EducationPerformance AssessmentPhysical ExaminationPsychometricsClinical AssessmentClinical Performance AssessmentSkills AssessmentStandardized ExaminationsTest ReliabilityAssessment Accuracy AnalysisAssessor ReliabilityInterrater ReliabilityMany-facet Rasch MeasurementObjective Structured AssessmentRasch Model AnalysisRater ErrorsStructured Clinical Assessment
The authors used a many-faceted Rasch measurement model to analyze rating data from a clinical skills assessment of 173 fourth-year medical students to investigate four types of rater errors: leniency, inconsistency, the halo effect, and restriction of range. Students performed six clinical tasks with 6 standardized patients (SPs) selected from a pool of 17 SPs. SPs rated the performance of each student in six skills: history taking, physical examination, interpersonal skills, communication technique, counseling skills, and physical examination etiquette. SPs showed statistically significant differences in their rating severity, indicating rater leniency error. Four SPs exhibited rating inconsistency. Four SPs restricted their ratings in high categories. Only 1 SP exhibited a halo effect. Administrators of objective structured clinical examinations should be vigilant for various types of rater errors and attempt to reduce or eliminate those errors to improve the validity of inferences based on objective structured clinical examination scores.
17
The assessment of clinical skills/competence/performance
George E. Miller · Academic Medicine · 1990 · 4.8K citations
Essentials of Psychological Testing
Benjamin Rosner, Lee J. Cronbach · The American Journal of Psychology · 1960 · 4.6K citations
Fundamentals of item response theory
Choice Reviews Online · 1992 · 3.5K citations
Education, Educational Assessment, Educational Measurement +17
Reasonable mean-square fit values
Benjamin G. Wright · Medical Entomology and Zoology · 1994 · 1.6K citations
J. P. Guilford · 1954 · 779 citations