Measuring pairwise agreement among many observers

Hubert J. A. Schouten

Biometrical Journal · 1980 · 49 citations · 8 references

Concepts

Abstract

Abstract Weighted kappa is defined as a measure of pairwise inter observer agreement. A weighted intra class kappa coefficient is proposed to measure agreement on a particular response category. An interclass kappa coefficient is proposed for each pair of response categories. Simple estimation procedures are presented for the case where the observers judging one subject are not necessarily the same as those judging another subject. Large sample standard errors are derived and a numerical example is given.

References

8