Statistical Methods in Assessing Agreement

TLDR

Agreement measurements are essential for evaluating new or generic processes across laboratory performance, validation, and bioequivalence, and recent methods—often assuming random target values—have emerged to capture data within meaningful boundaries. This review surveys agreement literature, introduces coverage‑probability–based methodologies, and compares their asymptotic power and trade‑offs. The authors derive analytical expressions for all measures under fixed or random targets with homogeneous or proportional error, illustrate them with data, and provide a downloadable SAS macro.

Abstract

Measurements of agreement are needed to assess the acceptability of a new or generic process, methodology, and formulation in areas of laboratory performance, instrument or assay validation, method comparisons, statistical process control, goodness of fit, and individual bioequivalence. In all of these areas, one needs measurements that capture a large proportion of data that are within a meaningful boundary from target values. Target values can be considered random (measured with error) or fixed (known), depending on the situation. Various meaningful measures to cope with such diverse and complex situations have become available only in the last decade. These measures often assume that the target values are random. This article reviews the literature and presents methodologies in terms of "coverage probability." In addition, analytical expressions are introduced for all of the aforementioned measurements when the target values are fixed and when the error structure is homogenous or heterogeneous (proportional to target values). This article compares the asymptotic power of accepting the agreement across all competing methods and discusses the pros and cons of each. Data when the target values are random or fixed are used for illustration. A SAS macro program to compute all of the proposed methods is available for download at http://www.uic.edu/~hedayat/.