Clarifying Question Meaning in a Household Telephone Survey

Abstract

This study contrasts two interviewing techniques that reflect different tacit assumptions about communication. In one, strictly standardized interviewing, interviewers leave the interpretation of questions up to respondents. In the other, conversational interviewing, interviewers say whatever it takes to make sure that questions are interpreted uniformly and as intended. Respondents from a national sample were interviewed twice. Each time they were asked the same factual questions from ongoing government surveys, five about housing and five about recent purchases. The first interview was strictly standardized; the second was standardized for half the respondents and conversational for the others. Respondents in a second conversational interview answered differently than in the first interview more often, and for reasons that conformed more closely to official definitions, than respondents in a second standardized interview. This suggests that conversational interviewing improved comprehension, although it also lengthened interviews. We conclude that respondents in a national sample may misinterpret certain questions frequently enough to compromise data quality and that such misunderstandings cannot easily be eliminated by pretesting and rewording questions alone. More standardized comprehension may require less standardized interviewer behavior.