Creativity Research Journal · 2008 · 364 citations · 32 references
MeasurementConsensual Assessment TechniqueDiagnosisEducationCognitionPsychometricsPsychologySocial SciencesCreativityPerformance AssessmentInterobserver AgreementPsychological EvaluationSurvey MethodologyAssessmentStatisticsReliabilityCognitive ScienceRehabilitationExperimental PsychologyComputational CreativityEvaluation MeasureCreativity AssessmentNonexpert RatersCreative Computing
The Consensual Assessment Technique (CAT) is one of the most highly regarded assessment tools in creativity, but it is often difficult and/or expensive to assemble the teams of experts required by the CAT. Some researchers have tried using nonexpert raters in their place, but the validity of replacing experts with nonexperts has not been adequately tested. Expert (n = 10) and nonexpert (n = 106) creativity ratings of 205 poems were compared and found to be quite different, making the simple replacement of experts by nonexpert raters suspect. Nonexpert raters' judgments of creativity were inconsistent (showing low interrater reliability) and did not match those of the expert raters. Implications are discussed, including the appropriate selection of expert raters for different kinds of creativity assessment.
32
Coefficient Alpha and the Internal Structure of Tests
Lee J. Cronbach · Psychometrika · 1951 · 42.3K citations · Full text