Using CATME to Document and Improve the Effectiveness of Teamwork in Capstone Courses

Abstract

Abstract This paper examines the use of peer evaluations and associated personal and team questions to diagnosis team functionality and provide early diagnosis of team dysfunctionalities. For a large STEM Capstone program in a major Midwestern university we collected peer evaluation data three times and examined both peer evaluation data and the relationships to the peer ratings of additional questions on teammates perception of psychological safety, team conflict, team cohesiveness, personal satisfaction with their team experience, and their written peer to peer comments explaining their peer ratings. We used the CATME peer evaluation system to collect the peer ratings and the additional questions data. CATME is a tool for forming and managing teams that has been used by over one million students to date. CATME's peer evaluation tool has five behaviorally anchored dimensions and additional team process measures that can be collected along with the peer evaluation data. We found strong correlations among team cohesiveness, team conflict and psychological safety questions. In addition, we developed heuristics by which problematic teams can be identified based on examining several of these team process scores simultaneously. In particular, psychological safety questions are strongly correlated negatively with team conflict feedback and positively correlated with team cohesion question feedback. These problem team indicators were strongly confirmed by the faculty mentoring our sample teams. A second useful team problem indicator was a lack of participation in the peer evaluation process. A final indicator of problematic teams was one or more low absolute levels of peer ratings for an individual team member and the nature of written peer to peer comments explaining the students' self and teammate peer ratings.