The weighing of evidence and the determinants of confidence

TLDR

Overconfidence and underconfidence in intuitive judgment arise when people emphasize the strength or extremeness of evidence while neglecting its weight or credibility. The authors propose that people’s confidence depends on the balance of arguments for and against competing hypotheses, with insufficient regard for evidence weight. They demonstrate this by first testing a chance setup where strength is defined by sample proportion and weight by sample size, then extending the analysis to complex evidential problems such as general knowledge questions and predicting self‑ and other‑behavior. The account explains how item difficulty drives overconfidence, links the gap between confidence judgments and frequency estimates to the illusion of validity, and refutes a frequentistic confidence model proposed by Gigerenzer and colleagues.

Abstract

The pattern of overconfidence and underconfidence observed in studies of intuitive judgment is explained by the hypothesis that people focus on the strength or extremeness of the available evidence (e.g., the warmth of a letter or the size of an effect) with insufficient regard for its weight or credence (e.g., the credibility of the writer or the size of the sample). This mode of judgment yields overconfidence when strength is high and weight is low, and underconfidence when strength is low and weight is high. We first demonstrate this phenomenon in a chance setup where strength is defined by sample proportion and weight is defined by sample size, and then extend the analysis to more complex evidential problems, including general knowledge questions and predicting the behavior of self and of others. We propose that people's confidence is determined by the balance of arguments for and against the competing hypotheses, with insufficient regard for the weight of the evidence. We show that this account can explain the effect of item difficulty on overconfidence, and we relate the observed discrepancy between confidence judgments and frequency estimates to the illusion of validity. Finally, we contrast the present account with a frequentistic model of confidence proposed by Gigerenzer and his colleagues, and present data that refute their model.