Does Conversational Interviewing Reduce Survey Measurement Error?

Abstract

Standardized survey interviewing is widely advocated in order to reduce interviewer-related error, for example by Fowler and Mangione. But Suchman and Jordan argue that standardized wording may decrease response accuracy because it prevents the conversational flexibility that respondents need in order to understand questions as survey designers intended. We propose that the arguments for these competing positions-standardized versus flexible interviewing approaches-may be correct under different circumstances. In particular, both standardized and flexible interviewing should produce high levels of accuracy when respondents have no doubts about how concepts in a question map onto their circumstances. However, flexible interviewing should produce higher response accuracy in cases where respondents are unsure about these mappings. We demonstrate this in a laboratory experiment in which professional telephone interviewers, using either standardized or flexible interviewing techniques, asked respondents questions from three large government surveys. Respondents answered on the basis of fictional descriptions so that we could measure response accuracy. The two interviewing techniques led to virtually perfect accuracy when the concepts in the questions clearly mapped onto the fictional situations. When the mapping was less