2013 · 14 citations · 11 references
ArchitectureInformation SystemsExperimental EconomicsHuman-computer InteractionUser Experience DesignAudience MeasurementUser ResearchCrowdsourcingQuantitative Evidence
To deliver voice over the Internet in a cost-effective way, it is essential to quantify the quality of user experience (i.e., QoE) of a voice service at various provisioning levels. Conducting user studies is an inevitable step facilitating quantitative studies of QoE. The two experimental methods -- lab experiment vs. crowdsourcing via Amazon Mechanical Turk [1] -- are compared in this study. We find that, for the study of Skype call quality, the crowdsourcing approach stands out in terms of efficiency and user diversity, which in turn strengthens the robustness and the depth of the analysis.
11
Michael D. Buhrmester, Tracy Kwang, Samuel D. Gosling · Perspectives on Psychological Science · 2011 · 10.1K citations
Quality management on Amazon Mechanical Turk
Panagiotis G. Ipeirotis, Foster Provost, Jing Wang · 2010 · 988 citations