Material Incentives and Other Potential Factors Associated With Response Rates to Internet Surveys of American Evaluation Association Members: Findings From a Randomized Experiment

Abstract

Internet surveys of American Evaluation Association (AEA) members are a common method for studying evaluation practice. Response rates obtained from Internet surveys of AEA members are, however, frequently very small. To investigate whether or not material incentives increase response rates to Internet surveys of AEA members, a between-subjects three-treatment and one control randomized experiment in which a randomly selected sample of AEA members were randomly assigned to a no-incentive control condition, lottery condition, token incentive condition, or philanthropic donation incentive condition was utilized. The overall response rate to the survey was 39.66% and the response rates for each of the four conditions were control = 36.24%, lottery = 44.39%, token incentive = 43.28%, and philanthropic donation = 34.67%, respectively. The cost-effectiveness of each of the four conditions also was examined, demonstrating that the lottery was the most cost-effective. Other factors potentially influencing response or nonresponse decisions also are discussed.