Design and Evaluation of an XML-Based Platform-Independent Computerized Adaptive Testing System

Abstract

This paper is an attempt to design and to evaluate the platform-independent computerized adaptive testing (CAT) system, which can expand the diversity of CAT-administering platforms. By using extensible markup language (XML) to describe the item bank, one might find the implementation of CAT on a different platform, such as a personal computer (PC), a personal digital assistant (PDA), and other handheld devices, more convenient. An experiment was conducted to examine the effects of a CAT administration platform on precision and efficiency. Fifty senior high school students were selected to take an English vocabulary CAT both on PC and PDA, which enabled them to compare the relevant advantages with the disadvantages of the two different administration platforms firsthand. Both tests used the same-size, well-calibrated item bank, ability estimation algorithm, and item selection strategy. The results indicate that the platforms on which examinees take CAT do not affect the performance of CAT. The responses of the questionnaire on the testing environment also show that most examinees prefer to take the test on PDA. It is concluded that using a PDA to administer CAT is both as precise and effective as a PC and more enjoyable and convenient.