Journal of Information Science · 2004 · 30 citations · 15 references
EducationInformation SystemsManagement Information SystemsPerformance MeasurementQuality AssuranceQuality ControlPeer AssessmentPeer ReviewReliabilityReview ProceduresAssessor ReliabilityConference Peer ReviewDouble-blind ReviewPeer Review EvaluationPeer-to-peer ReviewReviewer ReliabilityReviews
This paper analyses the reliability of the double-blind peer review systems used for submissions to the 2001 and 2002 UK Academy for Information Systems (UKAIS) conferences. The level of reliability found in the first conference was marginally lower than would be expected from a model based on chance. In the second conference the reliability level was significantly better, but still low. The paper explores some of the implications of this for the reviewing system, and suggests a model for assessing the impact of low levels of reliability.
15