Lab experiment vs. crowdsourcing

Yu-Chuan Yen, Cing-Yu Chu, Su‐Ling Yeh, Hao-Hua Chu, Polly Huang

2013 · 14 citations · 11 references

Concepts

Abstract

To deliver voice over the Internet in a cost-effective way, it is essential to quantify the quality of user experience (i.e., QoE) of a voice service at various provisioning levels. Conducting user studies is an inevitable step facilitating quantitative studies of QoE. The two experimental methods -- lab experiment vs. crowdsourcing via Amazon Mechanical Turk [1] -- are compared in this study. We find that, for the study of Skype call quality, the crowdsourcing approach stands out in terms of efficiency and user diversity, which in turn strengthens the robustness and the depth of the analysis.

References

11