Reliability and Validity of a Checklist to Evaluate Student Performance in a Problem-Based Learning Group

Abstract

<b>Objective.</b> To validate a problem-based learning (PBL) evaluation checklist to assess individual Doctor of Pharmacy (PharmD) students' performance in a group. <b>Methods.</b> In 2013, a performance checklist was developed and standardized. To evaluate the reliability and discriminant validity of the checklist, pharmacy students' evaluation scores from 2015-2016 were assessed along with overall program grade point averages (GPA), and scores on knowledge and problem-solving examinations. Predictive analysis software was used to analyze the data. <b>Results.</b> Seventy facilitators generated 1506 evaluation reports for 191 (90 third-year and 101 second-year) students over eight PBL cases. The mean (SD) total score was 40.6 (2.5) for P3s and 39.1 (2.7) for P2s out of a possible 44.2 points. Students' scores improved each semester. Interrater reliability based on intraclass correlation coefficient for all cases was 0.67. Internal reliability as determined by Cronbach alpha was >0.7 for all binary checklist items across all cases. Discriminant validity assessed using Pearson correlation coefficient showed that the total score from the checklist did not correlate with knowledge or problem-solving examination scores. <b>Conclusion.</b> This unique PBL checklist proved to be a reliable and valid tool to assess student performance in small group sessions in a PharmD curriculum.