Can human irrationality be experimentally demonstrated?

TLDR

The paper argues that recent studies on deductive and probabilistic reasoning do not imply human irrationality and classifies psychological research on the topic into four distinct categories. The authors assess fallacies using normative criteria rooted in shared intuitions, assert that competence theories must attribute rationality to ordinary people, and outline four experimental approaches: genuine cognitive errors, ignorance tests, misapplied criteria, and inappropriate criteria. They conclude that normative criteria are neither part of natural science nor provable by metamathematical proof, and that investigating performance errors can yield hypotheses about underlying information‑processing mechanisms.

Abstract

Abstract The object of this paper is to show why recent research in the psychology of deductive and probabilistic reasoning does not have "bleak implications for human rationality," as has sometimes been supposed. The presence of fallacies in reasoning is evaluated by referring to normative criteria which ultimately derive their own credentials from a systematisation of the intuitions that agree with them. These normative criteria cannot be taken, as some have suggested, to constitute a part of natural science, nor can they be established by metamathematical proof. Since a theory of competence has to predict the very same intuitions, it must ascribe rationality to ordinary people. Accordingly, psychological research on this topic falls into four categories. In the first, experimenters investigate conditions under which their subjects suffer from genuine cognitive illusions. The search for explanations of such performance errors may then generate hypotheses about the ways in which the relevant information-processing mechanisms operate. In the second category, experimenters investigate circumstances in which their subjects exhibit mathematical or scientific ignorance: these are tests of the subjects' intelligence or education. In the third and fourth categories, experimenters impute a fallacy where none exists, either because they are applying the relevant normative criteria in an inappropriate way or because the normative criteria being applied are not the appropriate ones.