Educational and Psychological Measurement · 1978 · 13 citations · 5 references
Public HealthEducational AssessmentEducational MeasurementBehavior Change TheoryExperimental DesignPsychometricsBehavior ChangeChange MetricsExperimental EvaluationModerator AnalysisBehavior Change EvaluationBehavior Change MeasurementChange Score ModelsError Level AnalysisError Variance EstimationPre-treatment
A Monte Carlo study was conducted to evaluate six models commonly used to evaluate change. The results revealed specific problems with each. Analysis of covariance and analysis of variance of residualized gain scores appeared to substantially and consistently overestimate the change effects. Multiple factor analysis of variance models utilizing pretest and post-test scores yield invalidly low F ratios. The analysis of variance of difference scores and the multiple factor analysis of variance using repeated measures were the only models which adequately controlled for pre-treatment differences, however, they appeared to be robust only when the error level was 50% or more. These results indicated that one of the persistent flaws of models traditionally used to evaluate change is that the generated statistics are confounded with the amount of error variance in the data. With this insight, a modification of the analysis of variance model was developed. A second Monte Carlo study was conducted to further evaluate the traditionally used models and the modified models. The results confirmed the findings of the first study and suggested that the modified model may be an adequate model with which change phenomena can be evaluated.
5
How we should measure "change": Or should we?
Lee J. Cronbach, Lita Furby · Psychological Bulletin · 1970 · 2.5K citations
Education, Educational Assessment, Educational Measurement +12
Quinn McNemar · Educational and Psychological Measurement · 1958 · 504 citations
Frederic M. Lord · Educational and Psychological Measurement · 1956 · 247 citations
Education, Educational Assessment, Educational Measurement +17