Can Appended Auxiliary Data be Used to Tailor the Offered Response Mode in Cross-Sectional Studies? Evidence from An Address-Based Sample

Abstract

Abstract In theory, offering each sample member the mode sequence that maximizes their response propensity should increase the response rate and/or reduce the amount of nonresponse follow-up relative to a design that offers all sample members the same mode sequence. However, for this sort of tailoring to be feasible in a cross-sectional survey, it must be possible to use data available prior to data collection (e.g., on the sampling frame) to predict sample members’ “mode-sensitivity”—the effect of the offered mode sequence on response propensity. Using data from randomized experiments incorporated into the 2016 and 2019 cycles of the National Household Education Survey, we evaluate whether data appended to an address-based sampling (ABS) frame can accurately predict the sensitivity of household-level response behavior to the initial offer of a paper questionnaire instead of a web instrument and whether a modeled-mode design that tailors the offered mode sequence (web-push vs. paper-only) based on the resulting predictions improves household-level data collection outcomes relative to a uniform web-push design. We find that several characteristics available on the ABS frame show statistically significant interactions with the offered mode sequence in determining the probability of response to initial survey mailings. Consequently, relative to a uniform web-push design, the modeled-mode design increased the response rate to early mailings and reduced the number of mailings required per response. However, the modeled-mode design did not meaningfully increase the final response rate, nor did it lead to a substantial reduction in indicators of nonresponse bias.