Financial incentives and the "performance of crowds"

TLDR

Financial incentives and performance have long intrigued social scientists, and the rise of web-based crowdsourcing has renewed interest. The study investigates how compensation affects performance in two Amazon Mechanical Turk experiments. The authors conduct two experiments on Amazon's Mechanical Turk to examine the effect of compensation on performance. Higher financial incentives raise the quantity but not the quality of AMT work, likely due to an anchoring effect, while a quota-based payment scheme yields better work at lower pay than a piece-rate scheme, findings that align with prior lab studies and suggest real-world implications.

Abstract

The relationship between financial incentives and performance, long of interest to social scientists, has gained new relevance with the advent of web-based "crowd-sourcing" models of production. Here we investigate the effect of compensation on performance in the context of two experiments, conducted on Amazon's Mechanical Turk (AMT). We find that increased financial incentives increase the quantity, but not the quality, of work performed by participants, where the difference appears to be due to an "anchoring" effect: workers who were paid more also perceived the value of their work to be greater, and thus were no more motivated than workers paid less. In contrast with compensation levels, we find the details of the compensation scheme do matter--specifically, a "quota" system results in better work for less pay than an equivalent "piece rate" system. Although counterintuitive, these findings are consistent with previous laboratory studies, and may have real-world analogs as well.