IX. On the problem of the most efficient tests of statistical hypotheses

TLDR

The problem of testing statistical hypotheses, rooted in Bayes' theorem, has been debated by scholars such as Bertrand and Borel, who illustrate differing views including Bertrand's pessimism that such tests cannot reliably confirm hypotheses. The common method involves computing the probability P that a chosen characteristic x of the observed data would occur if the hypothesis were true. A small P suggests the hypothesis is likely false, while a large P supports it, and Borel argued the method succeeds when the chosen characteristic is suitably remarkable.

Abstract

The problem of testing statistical hypotheses is an old one. Its origin is usually connected with the name of Thomas Bayes, who gave the well-known theorem on the probabilities a posteriori of the possible “causes" of a given event. Since then it has been discussed by many writers of whom we shall here mention two only, Bertrand and Borel, whose differing views serve well to illustrate the point from which we shall approach the subject. Bertrand put into statistical form a variety of hypotheses, as for example the hypothesis that a given group of stars with relatively small angular distances between them as seen from the earth, form a “system” or group in space. His method of attack, which is that in common use, consisted essentially in calculating the probability, P, that a certain character, x , of the observed facts would arise if the hypothesis tested were true. If P were very small, this would generally be considered as an indication that the hypothesis, H, was probably false, and vice versa . Bertrand expressed the pessimistic view that no test of this kind could give reliable results. Borel, however, in a later discussion, considered that the method described could be applied with success provided that the character, x , of the observed facts were properly chosen—were, in fact, a character which he terms “en quelque sorte remarquable.”