EducationEducational AssessmentEducational MeasurementCriterion-referenced AssessmentGeneralizability TheoryPsychometricsAcademic WritingAssessment ValidityReliabilityTest ReliabilityTest ValidityWriting AssessmentError Analysis In WritingGeneralizability Theory FacetsInter-observer AgreementTest Score Reliability
This study is an attempt at a cohesive characterization of the concept of essay reliability. As such, it takes as a basic premise that previous and current practices in reporting estimates for essay tests have certain shortcomings. The study provides an analysis of these shortcomings--partly to encourage a fuller understanding of the concept of as applied to essay testing, and partly to build the case that the framework of generalizabili'cy theory offers a much more satisfactory way of characterizing the concept. The study applies generalizability theory to existing research to illustrate that this approach is an improvement over the usual methods of estimating essay reliability. The paper also argues that the classical approach to has lead to a preoccupation with inter-marker agreement which '_ri turn has lead to a formulation of the reliability problem that makes it not susceptible of solution. Conceptual grounds for tol,...-sating inter-marker disagreement are advanced and the paper discusses conditions under which this may be so. However, the paper explains how generaliLability theory remains an appropriate framework for estimating the of essay scores whatever assumptions one chooses to make regarding inter-marker istency. cons
15
Statistical Methods for Research Workers
American Journal of Clinical Pathology · 1945 · 11.1K citations
Essentials of Psychological Testing
Benjamin Rosner, Lee J. Cronbach · The American Journal of Psychology · 1960 · 4.6K citations
Essentials of Educational Measurement
Alfred D. Garvin, Robert L. Ebel · Educational Researcher · 1980 · 1.5K citations
Education, Educational Assessment, Educational Measurement +17
Teaching and Assessing Writing
Barbara C. Karcher, Edward M. White · Teaching Sociology · 1986 · 304 citations