False Discovery Rate–Adjusted Multiple Confidence Intervals for Selected Parameters

TLDR

In applied research, confidence intervals are often reported only for parameters selected after data inspection. The study proposes extending the false discovery rate approach to selected confidence intervals, introducing the false coverage‑statement rate as a coverage metric. The authors present a procedure that adjusts the confidence level based on the number of parameters considered and selected, ensuring false coverage‑statement rate control and yielding intervals dual to the Benjamini–Hochberg testing procedure. The procedure achieves optimal intervals under independence, controls the false coverage‑statement rate under positive regression dependency for one‑ and two‑sided tests, extends to general dependency, and shows that Benjamini–Hochberg testing provides directional false discovery rate control.

Abstract

Often in applied research, confidence intervals (CIs) are constructed or reported only for parameters selected after viewing the data. We show that such selected intervals fail to provide the assumed coverage probability. By generalizing the false discovery rate (FDR) approach from multiple testing to selected multiple CIs, we suggest the false coverage-statement rate (FCR) as a measure of interval coverage following selection. A general procedure is then introduced, offering FCR control at level q under any selection rule. The procedure constructs a marginal CI for each selected parameter, but instead of the confidence level 1 − q being used marginally, q is divided by the number of parameters considered and multiplied by the number selected. If we further use the FDR controlling testing procedure of Benjamini and Hochberg for selecting the parameters, the newly suggested procedure offers CIs that are dual to the testing procedure and are shown to be optimal in the independent case. Under the positive regression dependency condition of Benjamini and Yekutieli, the FCR is controlled for one-sided tests and CIs, as well as for a modification for two-sided testing. Results for general dependency are also given. Finally, using the equivalence of the CIs to testing, we prove that the procedure of Benjamini and Hochberg offers directional FDR control as conjectured.