A Framework for Analyzing Feedback in a Formative Assessment System for Mathematical Modeling Problems

Abstract

B ackground Providing formative feedback on student responses to complex, open‐ended problems is challenging as solutions vary in content and quality. Instructors need to interpret student work and give feedback with high potential to guide students to close the gap between actual and reference level performance. To understand instructor feedback and its impact, a framework for analyzing instructor feedback is necessary. P urpose (H ypothesis ) The purposes of this study were to develop a framework to analyze instructor feedback on team responses to open‐ended mathematical modeling problems and to demonstrate the use of the framework in investigating formative assessment systems. D esign /M ethod To develop the framework, formative feedback from graduate teaching assistants on first‐year engineering student team responses to the Just‐In Time Manufacturing Model‐Eliciting Activity (MEA) was analyzed. The framework in conjunction with a four dimensional MEA Rubric was then used to identify patterns in feedback. R esults Feedback has both form and substance. Four forms of feedback were identified: comments not intended to prompt change; and questions, open suggestions, and direct suggestions intended to prompt change. The substance was tied to the criteria used to evaluate MEA responses. Results of applying the framework yielded two claims: (1) the rubric criteria influence the form and intent of feedback, and (2) the perceived quality of student work also impacts the form and intent of feedback. C onclusion The framework that emerged from this work is a useful tool for investigating instructor feedback in this context. Further, the framework is believed to be extendable to other learning contexts.