A Comparison of Two, Three and Four-Choice Item Tests Given a Fixed Total Number of Choices

Abstract

While measurement textbooks typically recommend the use of four or five alternatives with multiple-choice test items theoretical work by Tversky (1964) and some empirical studies indicate that three-choice items may be optimal under certain circumstances. In this study the characteristics of tests composed entirely of two, three or four-choice items were investigated given a fixed total number of alternatives across the whole test (Tversky's condition). Their relative merits were also estimated after allowing for differences in testing time. The results showed that number of alternatives per item was inversely related to item difficulty but directly related to item discrimination. Reliability and standard error of measurement of three-choice item tests was equivalent or superior to tests of four or two-choice items and these results held up after taking account of testing time. The use of three-choice items in typical classroom settings is recommended.