Descriptive data analysis: a concept between confirmatory and exploratory data analysis.

Abstract

Confirmatory Data Analysis (CDA) in randomized comparative (“controlled”) studies with many variables and/or time points of interest finds its limitations in the multiplicity of desired inferential statements which leads to unfeasibly small adjusted significance levels (“Bon-ferronization”) and, thereby, to unduly increased risks of not rejecting false hypotheses. In general, analytical models adequate for such complex data structures and suitable for practical use do not exist as yet. Exploratory Data Analysis (EDA), on the other hand, is usually intended to generate hypotheses and not to lead to final conclusions based on the results of the study. In this paper, it is proposed to fill the conceptual gap between CDA and EDA by “Descriptive Data Analysis” (“DDA”) which concept is mainly based on descriptive inferential statements. The results of a DDA in a controlled study are interpreted simultaneously on the basis of the investigator’s experience with respect to numerically relevant treatment effect differences and on “descriptive significances” as they appear in “near regular” patterns corresponding to the resulting relevant effect differences. A DDA may also contain confirmatory parts and/or tests on global hypotheses at a prechosen maximum risk α of erroneously rejecting true hypotheses. The paper is in parts expository and is addressed to investigators as well as statisticians.