A test of the “messages‐as‐fixed‐effect fallacy” argument: Empirical and theoretical implications of design choices

Abstract

Jackson and Jacobs (1983) reopened a debate, already long resolved by statisticians and linguists, that dealt with treating language variables as random versus fixed effects in analysis of variance designs. Jackson and Jacobs extended the argument to suggest that designs were invalid unless they included multiple messages within single experiments. An ensuing debate has filled scarce journal space, often advancing specious arguments, but at times offering sensible alternatives to the position that there is an invariant messages‐as‐fixed‐effect fallacy in our literature. Despite the lengthy debate, there has been no empirical scrutiny of relevant claims. Presented here is the first empirical test of three alternative analytic models using data from a carefully controlled experiment.