Evaluation Criteria for Discrete Confidence Intervals

Abstract

Confidence intervals for discrete distributions are often evaluated only by coverage and expected length. We discuss two additional criteria, p-confidence and p-bias. The choice of these criteria is motivated by the interpretation of a confidence interval as being the set of parameter values not rejected by a hypothesis test. Using these additional criteria we compare a number of equal-tailed confidence intervals for the binomial distribution. It is shown that methods that produce superior intervals, as measured by coverage and length, need not perform well in terms of p-confidence and p-bias. Cox's measuring device example is discussed to motivate the need for criteria beyond coverage and length.