Expert Status and Performance

TLDR

Expert judgments are crucial in time‑constrained or novel decision contexts, where good advice can save lives and money, and experts are typically defined by qualifications, track record, and experience, with the social expectation hypothesis suggesting that higher regard and experience predict better advice. The study asked experts to forecast their own and their peers’ performance on question sets, thereby examining the relationship between perceived and actual expertise. Results show that while experts’ mutual assessments are consistent, rank alone poorly predicts actual performance, and that expert advice improves when technical decisions employ broad expert groups, structured protocols, and feedback.

Abstract

Expert judgements are essential when time and resources are stretched or we face novel dilemmas requiring fast solutions. Good advice can save lives and large sums of money. Typically, experts are defined by their qualifications, track record and experience [1], [2]. The social expectation hypothesis argues that more highly regarded and more experienced experts will give better advice. We asked experts to predict how they will perform, and how their peers will perform, on sets of questions. The results indicate that the way experts regard each other is consistent, but unfortunately, ranks are a poor guide to actual performance. Expert advice will be more accurate if technical decisions routinely use broadly-defined expert groups, structured question protocols and feedback.