Applied Measurement in Education · 2011 · 132 citations · 45 references
EducationEducational MeasurementScience EducationPsychometricsScience AssessmentScience InstructionItem AnalysisScience IntegrationTest ValidityValidity EvidenceCohort Comparison MetricsConstruct ValidityConstruct Validity AssessmentContent Validity EvidenceMultiple-choice ItemsMultiple-choice Science ItemsRasch Model Analysis
This study explores measurement of a construct called knowledge integration in science using multiple-choice and explanation items. We use construct and instructional validity evidence to examine the role multiple-choice and explanation items plays in measuring students' knowledge integration ability. For construct validity, we analyze item properties such as alignment, discrimination, and target range on the knowledge integration scale using a Rasch Partial Credit Model analysis. For instructional validity, we test the sensitivity of multiple-choice and explanation items to knowledge integration instruction using a cohort comparison design. Results show that (1) one third of correct multiple-choice responses are aligned with higher levels of knowledge integration while three quarters of incorrect multiple-choice responses are aligned with lower levels of knowledge integration, (2) explanation items discriminate between high and low knowledge integration ability students much more effectively than multiple-choice items, (3) explanation items measure a wider range of knowledge integration levels than multiple-choice items, and (4) explanation items are more sensitive to knowledge integration instruction than multiple-choice items.
45
A Rasch Model for Partial Credit Scoring
Geoff N Masters · Psychometrika · 1982 · 3.8K citations
Eliciting self-explanations improves understanding,
Cognitive Science · 1994 · 2.1K citations
The Role of Assessment in a Learning Culture
Lorrie A. Shepard · Educational Researcher · 2000 · 2K citations
National Science Education Standards
Choice Reviews Online · 1996 · 1.9K citations
Lincoln E. Moses, E. F. Lindquist · Journal of the American Statistical Association · 1951 · 1.7K citations