The Expected Value of Information and the Probability of Surprise

Abstract

Risk assessors attempting to use probabilistic approaches to describe uncertaintyoften find themselves in a data‐sparse situation: available data are only partially relevant to the parameter of interest, so one needs to adjust empirical distributions, use explicit judgmental distributions, or collect new data. In determining whether ornot to collect additional data, whether by measurement or by elicitation of experts, it is useful to consider the expected value of the additional information. The expected value of information depends on the prior distribution used to represent current information; if the prior distribution is too narrow, in many risk‐analytic cases the calculated expected value of information will be biased downward. The well‐documentedtendency toward overconfidence, including the neglect of potential surprise, suggeststhis bias may be substantial. We examine the expected value of information, includingthe role of surprise, test for bias in estimating the expected value of information, and suggest procedures to guard against overconfidence and underestimation of the expected value of information when developing prior distributions and when combining distributions obtained from multiple experts. The methods are illustrated with applications to potential carcinogens in food, commercial energy demand, and global climate change.