Publication | Closed Access
Quantitative analysis of the rubric as an assessment tool: an empirical study of student peer‐group rating
257
Citations
0
References
2003
Year
EngineeringGroup AssessmentGeneralizability TheoryEducationClassical Test TheoryProgram EvaluationAcademic StrengthPerformance AssessmentQuantitative AnalysisApplied MeasurementClassroom AssessmentSurvey MethodologyReliabilityCollege Biology StudentsStudent Peer‐group RatingEducational TestingEducational MeasurementGradingPerformance StudiesStudent AssessmentHigher Education AssessmentEducational EvaluationEducational AssessmentAssessment Tool
The rubric is widely used in progressive education, yet little evidence exists on its effectiveness when applied by students. The study aims to evaluate the validity and reliability of the rubric for student peer‑group assessment. The authors collected 1,577 rubric ratings from 107 biology students over three years during oral presentations. The analysis demonstrates that students and instructors use the rubric consistently, that it is gender‑neutral and unaffected by academic strength, that student and instructor scores align perfectly, and that inter‑rater reliability is moderate, confirming the rubric’s validity and usefulness for peer assessment.
Abstract Although the rubric has emerged as one of the most popular assessment tools in progressive educational programs, there is an unfortunate dearth of information in the literature quantifying the actual effectiveness of the rubric as an assessment tool in the hands of the students. This study focuses on the validity and reliability of the rubric as an assessment tool for student peer‐group evaluation in an effort to further explore the use and effectiveness of the rubric. A total of 1577 peer‐group ratings using a rubric for an oral presentation was used in this 3‐year study involving 107 college biology students. A quantitative analysis of the rubric used in this study shows that it is used consistently by both students and the instructor across the study years. Moreover, the rubric appears to be 'gender neutral' and the students' academic strength has no significant bearing on the way that they employ the rubric. A significant, one‐to‐one relationship (slope = 1.0) between the instructor's assessment and the students' rating is seen across all years using the rubric. A generalizability study yields estimates of inter‐rater reliability of moderate values across all years and allows for the estimation of variance components. Taken together, these data indicate that the general form and evaluative criteria of the rubric are clear and that the rubric is a useful assessment tool for peer‐group (and self‐) assessment by students. To our knowledge, these data provide the first statistical documentation of the validity and reliability of the rubric for student peer‐group assessment.