Comparison of the reliability and validity of scores from two concept-mapping techniques

TLDR

The study compared a high‑directed fill‑in‑the‑map technique with a low‑directed construct‑a‑map technique to assess sensitivity to sample, equivalence, and the extent to which each method reveals students’ connected understanding. Results showed that fill‑in‑the‑map scores were insensitive to the sample, the two fill‑in variants were not equivalent, and high‑directed maps overestimated performance while low‑directed maps better differentiated students’ knowledge structures. © 2001 John Wiley & Sons, Inc., J Res Sci Teach 38:260–278.

Abstract

This paper reports the results of a study that compared two concept-mapping techniques, one high-directed, “fill-in-the-map,” and one low-directed, “construct-a-map-from-scratch.” We examined whether: (1) skeleton map scores were sensitive to the sample of nodes or linking lines to be filled in; (2) the two types of skeleton maps were equivalent; and (3) the two mapping techniques provided similar information about students' connected understanding. Results indicated that fill-in-the-map scores were not sensitive to the sample of concepts or linking lines to be filled in. Nevertheless, the fill-in-the-nodes and fill-in-the-lines techniques were not equivalent forms of fill-in-the-map. Finally, high-directed and low-directed maps led to different interpretations about students' knowledge structure. Whereas scores obtained under the high-directed technique indicated that students' performance was close to the maximum possible, the scores obtained with the low-directed technique revealed that students' knowledge was incomplete compared to a criterion map. We concluded that the construct-a-map technique better reflected differences among students' knowledge structure. © 2001 John Wiley & Sons, Inc. J Res Sci Teach 38: 260–278, 2001