Assessment & Evaluation in Higher Education · 2001 · 55 citations · 8 references
EducationEducational AssessmentHigher EducationFormative AssessmentLarge-scale AssessmentTest ConstructionAutomated Essay ScoringComputer-based TestingEducational Software EvaluationOnline AssessmentStudent AssessmentWriting AssessmentCycle TimeHuman Scoring ComparisonObserved VariablesPlacementStatistical Model SpecificationWriting Placement AssessmentWritten Examinations
Abstract This study examined the feasibility of employing Project Essay Grade (PEG) software to evaluate web-based student essays that serve as placement tests at a large Mid-Western university. The results of two experiments are reported. In the first experiment, the essays of 1293 high school and college students were used to create a statistical model for the PEG software. PEG identified 30 proxes (observed variables) that could be incorporated into an evaluation of the written work. In the second experiment, the ratings from a separate sample of 617 essays were used to compare the ratings of six human judges against those generated by the computer. The inter-judge correlation of the human raters was r = 0.62 and was r = 0.71 for the computer. Finally, the PEG software was an efficient means for grading the essays with a capacity for approximately three documents graded every second. Cycle time from the websubmission of the document to producing a report score was about 2 minutes. Although PEG would appear to be a cost-effective means of grading written work of this type, several cautionary notes are included.
8
Teaching and Assessing Writing
Jean-Marie Schultz, Edward M. White · Modern Language Journal · 1996 · 76 citations