The Development and Application of an Instrument for Assessing Resident Competence During Preanesthesia Consultation

Abstract

In Brief In this study, we aimed to construct, validate, and apply an instrument for assessing resident performance at outpatient preanesthesia consultation (PAC). A focus group and a Delphi panel of experts defined component items of a typical outpatient PAC, which could be used as indicators of competence. Items were incorporated in a checklist, which was further validated in a sample of consultations performed by board-certified anesthesiologists. The resulting instrument contained 37 items, grouped into five domains (physician-patient relationship, medical history, physical examination, patient education, and preanesthesia records), with high construct validity, high discriminant validity, moderate internal consistency, and high probability of inter-raters agreement. The instrument was applied to evaluate the performance of seven first-year residents at 317 consecutive PAC. Data were analyzed by constructing exponentially weighted moving average charts for domain and total scores. Statistically significant differing levels of performance could be consistently detected. Applying exponentially weighted moving average charts to the sequential analysis of the developed checklist scores can reliably assess resident performance at the devised criteria. The Preanesthesia Consultation Scoring Checklist is a potentially useful instrument for both formative and summative assessment of residents during their training in processes involved in outpatient preanesthesia evaluation. IMPLICATIONS: A resident assessment score at outpatient preanesthesia consultation was constructed, validated, and applied. Quality assurance statistics were used for its interpretation. The instrument proved useful in scoring resident performance and in identifying different patterns of performance.