Analysis of the Astronomy Diagnostic Test

Abstract

Seventy undergraduate class sections were examined from the database of Astronomy Diagnostic Test
\n(ADT) results of Deming and Hufnagel to determine if course format correlated with ADT normalized
\ngain scores. Normalized gains were calculated for four different classroom scenarios: lecture, lecture with
\ndiscussion, lecture with lab, and lecture with both lab and discussion. Statistical analysis shows that there
\nare no significant differences in normalized gain among the self- reported classroom formats. Prerequisites
\nrelated to mathematics courses did show differences in normalized gain. Of all reported course activities,
\nonly the lecture and the readings for the course correlate significantly with the normalized gain. This
\nanalysis suggests that the ADT may not have enough sensitivity to measure differences in the
\neffectiveness of different course formats because of the wide range of topics that the ADT addresses with
\nfew questions. Different measures of gain and their biases are discussed. We argue that the use of the
\nnormalized gain is not always warranted because of its strong bias toward high pretest scores.