User interface design for crowdsourcing systems

Abstract

Harnessing human computation through crowdsourcing offers an alternative approach to solving complex problems, especially those that are relatively easy for humans but difficult for computers. Micro-tasking platforms such as Amazon Mechanical Turk have attracted large, on-demand work force of millions of workers as well as hundreds of thousands of job requesters. Achieving high quality results by putting humans in the loop is one of the main goals of these crowdsourcing systems. We study the effects of different user interface designs on the performance of crowdsourcing systems. Our results indicate that user interface design choices have a significant effect on crowdsourced worker performance.