Candidate Surveys on Program Evaluation: Examining Instrument Reliability, Validity and Program Effectiveness

Abstract

This self-study conducted by an urban college of education examined the effectiveness of its teacher education programs in emphasizing its social justice conceptual framework, as perceived by its teacher candidates. The study included development of an electronic survey based on the ten learning outcomes of the unit’s conceptual framework, and testing the internal consistency reliability and the construct validity of the survey items. The survey was administered to teacher education candidates who rated their program’s effectiveness in fulfilling the unit’s vision of preparing educators that are informed and empowered, committed, and engaged with students and their communities. We analyzed data from teacher candidates at mid or end points in their programs over two academic years. Our results indicated that the survey was a valid and reliable instrument for collecting candidates’ feedback on program improvement. Candidates’ feedback showed differences among the three programs in their emphasis on the conceptual framework. Additionally, there were some learning outcomes that were rated low in both years by the candidates, implying that some outcomes are being more successfully emphasized in the programs than others. These results were important feedback for the teacher education unit to move along the path of self-reflection and continuous improvement.