A Generalization of Cohen's Kappa Agreement Measure to Interval Measurement and Multiple Raters

Abstract

Cohen's kappa statistic is frequently used to measure agreement between two observers employing categorical polytomies. In this paper, Cohen's statistic is shown to be inherently multivariate in nature; it is expanded to analyze ordinal and interval data; and it is extended to more than two observers. A nonasymptotic test of significance is provided for the generalized statistic.