The testing of listening comprehension: an introspective study1

TLDR

The study uses verbal reports to investigate how listening tests function and how normally inaccessible processes affect test performance. The authors collected introspective data from six participants on four main areas—short‑answer test effects, higher‑level cognition measurement, monitoring of interpretation, and question preview influence—and found that listening comprehension is an inferential process beyond linguistic knowledge, posing a dilemma for testers. The protocols show that there are no clear objective criteria for judging interpretation appropriateness, and the paper discusses the implications of this dilemma.

Abstract

The present research uses the verbal report methodology to examine how listening tests work, and how processes not normally accessible through quan titative research methods influence test performance. Six introspectees pro vided data on four main areas of interest: the influence of the short-answer test method on the measurement of listening comprehension; whether test items can measure 'higher-level' cognitive processes; whether test items can measure how well listeners monitor the appropriacy of their interpretation; and how question preview influences comprehension and test performance. The interview protocols provide a great deal of data relevant to these and related issues, the presentation and interpretation of which is the main purpose of this paper. The protocols also indicate a serious dilemma for language testers in that listening comprehension involves far more than the application of linguistic knowledge to produce a propositional representation of a text; rather it is an inferential process in which listeners attempt to construct an interpretation which is meaningful in the light of their own assessment of the situation, knowledge and experience. Thus there are often no clear, objective criteria against which to judge the appropriacy of any one interpretation. The implications of this are discussed.