Wrong: Why Experts* Keep Failing Us - and How to Know When Not to Trust Them

Abstract

Wrong: Why Experts* Keep Failing Us - And How to Know When Not to Trust Them. David H. Freedman Little, Brown and Company, 2010 Much time and effort in the biomedical sciences is spent on understanding the mechanism of diseases. Based on this understanding, scientists try to influence the course of diseases and restore health by developing mechanism-based treatments. Thus, significant amounts of resources are invested in developing disease models to both uncover mechanisms and to evaluate potential therapeutics. As with any human endeavor, the underlying premise of the model and the interpretation of data generated by such models are sensitive to human cognitive limitations. As a result of these limitations, perhaps all too often, experimental treatments that show promise in the laboratory fizzle in clinical trials and the prevailing understanding of health and disease comes under question. Journalist David Freedman seeks to understand why experts go wrong. Although Freedman aims to understand why experts in general have failed to give accurate predictions based on accumulated data of past events, experts in biomedical research come under the harshest scrutiny. accounts of the failings of nonscientists, for example, of economists, financial and business experts are rather superficial in the book. Better accounts can be found elsewhere (e.g. Murphey, 2009; Woods, 2011). Being a biomedical scientist, I skipped ahead to the chapter The Trouble with Scientists, Part 1 (there are two parts). Freeman's review of the general state of the field is quite sobering but while I found myself nodding in agreement, I also felt slightly peeved that objections to the lack of consistent success in biomedical research would come from a non-scientist. Surely the scientific community is scrupulously self-correcting and there are internal mechanisms that purge weak hypotheses supported by flimsy data. However, self-correction is hardly the rule. In fact, in The Trouble with Scientists, Part 2, the impression one is left with is that it is not in the scientists best interest to point out flawed or incorrect research, rather, to focus on the production of interesting and positive data, which in turn may be flawed as well. Since researchers are fully occupied in trying to compete for limited federal funds, few researchers have the time and resources to replicate others' studies. Even if such work is undertaken, Freedman points out that scientific journals are not be highly inclined to publish the work, particularly if findings go against the prevailing consensus. Thus, such data are confined to a file-drawer, to never to see the light of day, which leads to publication bias - and skewed support in favor of an hypothesis. Rather than being tested by contrary data or alternate explanation, established hypotheses become so merely by default. Although Freedman suggests a number of contributing factors that have tarnished the reputation of modern biomedical research, at the core is human nature. Stiff competition for limited public money to fund academic laboratories and salaries lead to a do-whatever-is necessary mindset and politicking. In addition, the peer review system for grants as well as publications is not immune to individual biases and personalities. A scientist sums up the psychology of his colleagues: ... beliefs... are shaped by 'all of the vanities, vested interests, hunches, experiences, politics, careerism, grantsmanship tactics, competing cadres of collaborators, imperfections, and backgrounds'... Thus, in contrast to the popular vision of an idealist getting at the truth, scientists share traits that are no different from nonscientists needed in order to succeed. As mentioned earlier, the book spends a considerable amount of time describes failings of scientists, but the overall theme can be applied to those for whom we depend on for expert advice. Freeman attempts to examine the psychology of experts, but his evaluation is superficial at best and the topic could fill another book. …