A weighted kappa statistic for reliability testing in performance analysis of sport

Abstract

The kappa statistic has been used to evaluate the reliability of performance indicators measured using computerised notational analysis systems. However, all disagreements between observers are treated as total disagreements event if neighbouring values of ordinal scale performance indicators are used. Where nominal values are used to represent areas of the playing surface, kappa does not give any credit where the observers record neighbouring cells. Therefore, the purpose of the current paper was to illustrate how the weighted kappa statistic can be used with performance analysis data to give some credit where there are partial agreements between the observers.