Proceedings of the AAAI Conference on Artificial Intelligence · 2016 · 36 citations · 25 references
Social SciencesEconomicsExperimental DesignExperimental EconomicsGame TheoryMechanism DesignPeer ReviewReputationCrowdsourcingCrowdsourcing Decision-makingPeer Review MechanismsPeer-to-peer ReviewStrategyproof Mechanisms
We study an important crowdsourcing setting where agents evaluate one another and, based on these evaluations, a subset of agents are selected. This setting is ubiquitous when peer review is used for distributing awards in a team, allocating funding to scientists, and selecting publications for conferences. The fundamental challenge when applying crowdsourcing in these settings is that agents may misreport their reviews of others to increase their chances of being selected. We propose a new strategyproof (impartial) mechanism called Dollar Partition that satisfies desirable axiomatic properties. We then show, using a detailed experiment with parameter values derived from target real world domains, that our mechanism performs better on average, and in the worst case, than other strategyproof mechanisms in the literature.
25
C. L. Mallows · Biometrika · 1957 · 782 citations
Equity: In Theory and Practice
Robert E. Kuenne, H. Peyton Young · Southern Economic Journal · 1995 · 682 citations
Tuned Models of Peer Assessment in MOOCs
Chris Piech, Jonathan Huang, Zhenghao Chen et al. · arXiv (Cornell University) · 2013 · 241 citations · Full text
Education, Educational Assessment, Educational Technology +17
Learning Mallows Models with Pairwise Preferences
Tyler Lu, Craig Boutilier · 2011 · 160 citations