Eliciting Factor Importance in a Designed Experiment

Abstract

Recently, there has been great interest in the Bayes model for analyzing confounded designs. This model suggests that only a few of the main effects and interactions are "active" and estimates the posterior probability that a given factor is active. This article proposes using pairwise comparisons to elicit an expert's opinion and form a well-defined, coherent prior. The prior probability that a factor is active is modeled as a "preference" in the Bradley–Terry linear model for pairwise comparisons. This article provides suggested schedules that minimize the number of comparisons offered to the expert based on the expression of a comparison schedule as a graph theory problem. Examples demonstrate that an expert's knowledge can be obtained to adequate precision for the Bayes analysis of screening designs by asking a few simple questions.