Classical meets modern in the IDEA protocol for structured expert judgement

Abstract

Expert judgement is pervasive in all forms of risk analysis, yet the development of tools to deal with such judgements in a repeatable and transparent fashion is relatively recent. This work outlines new findings related to an approach to expert elicitation termed the IDEA protocol. IDEA combines psychologically robust interactions among experts with mathematical aggregation of individual estimates. In particular, this research explores whether communication among experts adversely effects the reliability of group estimates. Using data from estimates of the outcomes of geopolitical events, we find that loss of independence is relatively modest and it is compensated by improvements in group accuracy.