League Tables and Their Limitations: Statistical Issues in Comparisons of Institutional Performance

TLDR

Increasing interest in public‑institution accountability has prompted the need for quantitative comparisons of health and education institutions. This paper aims to outline the statistical issues involved in making such quantitative comparisons. The authors detail how model‑based uncertainty must be incorporated, appropriate outcome and baseline measures defined, and careful interpretation of apparent differences exercised. They highlight that statistical methods can illuminate the extent and possible causes of institutional differences, while cautioning about the limitations of these methods.

Abstract

SUMMARY In the light of an increasing interest in the accountability of public institutions, this paper sets out the statistical issues involved in making quantitative comparisons between institutions in the areas of health and education. We deal in detail with the need to take account of model-based uncertainty in making comparisons. We discuss the need to establish appropriate measures of institutional 'outcomes' and base-line measures and the need to exercise care and sensitivity when interpreting apparent differences. The paper emphasizes that statistical methods exist which can contribute to an understanding of the extent and possible reasons for differences between institutions. It also urges caution by discussing the limitations of such methods.