Assessment & Evaluation in Higher Education · 2019 · 21 citations · 38 references
EducationEducational AssessmentEducational MeasurementAssessment FairnessClassroom AssessmentItem Response TheoryBiasQuality Of EducationResponse BiasStudent AssessmentTeacher EvaluationTest BiasBias In Educational TestingBias In Teacher EvaluationLatent Regression ModelsProcess QualityRasch Rating Scale ModelRating Scale AnalysisStudent Evaluation Of TeachingTeacher Assessment Validity
The validity of student evaluation of teaching (SET) scores depends on minimum effect of extraneous response processes or biases. A bias may increase or decrease scores and change the relationship with other variables. In contrast, SET literature defines bias as an irrelevant variable correlated with SET scores, and among many, a relevant biasing factor in literature is the instructor’s gender. The study examines the extent to which acquiescence, the tendency to endorse the highest response option across items and bias in the first sense affects students’ responses to a SET rating scale. The study also explores how acquiescence affects the difference in teaching quality (TQ) by instructor’s gender, a bias in the latter sense. SET data collected at a faculty of education in Ontario, Canada were analysed using the Rasch rating scale model. Findings provide empirical support for acquiescence affecting students’ responses. Latent regression analyses show how acquiescence reduces the difference in TQ by instructor’s gender. Findings encourage greater attention to the response process quality as a way to better defend the utility of SET and prevent potentially misleading conclusions from the analysis of SET data.
38
Statistical Power Analysis for the Behavioral Sciences
Keith E. Muller, Jacob Cohen · Technometrics · 1989 · 83.9K citations
Judith P. Frankmann, Jeff Adams · The American Journal of Psychology · 1904 · 4.3K citations
A constant error in psychological ratings.
Edward L. Thorndike · Journal of Applied Psychology · 1920 · 2.1K citations