This is another post about how non-physicians can understand how physicians use evidence. As I noted before, medical evidence has a hierarchy of reliability. The least reliable of these is expert opinion. This seems counter-intuitive: why is expert opinion the worst sort of evidence? Should not the experts know what they are talking about? In general, of course, experts do know what they are talking about — that is what makes them experts. But a closer examination of the matter shows why this kind of evidence is the weakest and most subjective; after all, it is one person’s opinion (or sometimes a committee of persons’ opinions), and opinions can be incorrect.
We need to look closely at just why a particular individual is considered an expert, and by whom. Credentials are important: where did the person obtain her training, where does she work now, and what is her standing among her peers? These seem obvious questions to ask, but these days an astonishing number of people with dubious or no credentials can write a book, put up a website, or start a blog, and, if they are persuasive marketers, can convince others they are experts. Fortunately for parents, the same wide-open quality of the internet allows one to search the background, credentials, and accomplishments of any putative expert. Wise parents will do this as a matter of course before deciding whether to take the expert’s advice on any important matter about their child’s health.
Experts who advise you to do one thing or another with your child typically base their advice to you on their own interpretation of the available medical research. They have the knowledge and training to understand the often esoteric medical literature. In addition, most experts themselves do research in the relevant field. Those are important and useful things. However, there are still good reasons why we should regard such opinions as the worst kind of data — better than nothing, but sometimes only barely so.
For one thing, the reason an expert holds a particular opinion may be because she was taught that way by her teachers, who may have, in turn, been taught the same thing by their teachers. Medicine is practical and empirical enough that such received, traditional opinions will not be tossed out unless they are consistently wrong. It is also true we physicians venerate our medical forebears to the extent that misguided opinions can occasionally persist long after they should have been discarded. So sometimes the answer to why doctors do something a particular way is that we have always done it that way.
Experts also form their opinions based upon what they have seen in the past. If their experience is long, often they have seen quite a few instances of whatever is under discussion, and that experience should count for something. On the other hand, memory is a tricky thing; sometimes we recall things in ways that can ultimately prove misleading. For example, the more striking and dramatic things tend to stay in our minds better than the more mundane things, and medical experts are not immune to this phenomenon. For example, I know that I remember unusual manifestations of certain cases for decades, and this inevitably colors how I approach the next child with that particular problem. Even though I know the case was unusual, I naturally think of its circumstances whenever I care for another child with whatever the disorder was. This is an example of what we call recall bias.
There are other kinds of bias that may affect the judgment of medical experts, and some of them are not innocent things like tricks of memory. Medical experts are no different from other kinds of experts, such as foreign policy pundits or stock market analysts, in that we, too, may have agendas that are not obvious to parents listening to our advice. Any controversial subject will lead to partisanship, and medical debates are no different. Parents considering the advice of medical experts should be alert to what a particular expert’s agenda might be. This is not necessarily a sinister thing; I think the great majority of experts advocating one position or another do so because they truly believe it is the correct one. But it is still a real thing.
Sometimes, however, medical experts may have agendas that are not so innocent. For example, there have been recent examples of experts touting one treatment over another when they have an undisclosed interest in the outcome. The conflict of interest could be intellectual, such as past friendships or associations with researchers of a particular treatment, or they could be crassly commercial. The ethical boundaries are, in theory, quite sharp, but there have been well-known examples of medical experts recommending a particular treatment when they have a financial conflict of interest regarding the outcome.
The best experts, of course, are the ones who tell you the basis of their opinions. They do not just pontificate about what is best for your child — they tell you how their reading of the best evidence led them to their informed opinion. They interpret for you the meaning of the scientific evidence. On the whole, most medical experts will do this for parents. Even so, it is a good thing for parents not to take whatever the experts say entirely on faith; it is better to have some grasp of how medical evidence is collected and analyzed. And besides, even experts can be wrong.
11/03/2008 • This is another post about how non-physicians can understand how physicians use evidence. As I noted before, medical evidence has a hierarchy of reliability. The least reliable of these is expert opinion. This seems counter-intuitive: ...more
05/23/2011 • Over the next month I'm going to repost a series I did about how to understand medical evidence -- how to read the medical news. How can a non-physician interpret the latest breathless bulletin about ...more
07/17/2011 • Here is another post in understanding the proper use of medical evidence. (Previous ones in the series are here, here, here, and here) Medical researchers can use several techniques to try and get around some ...more
07/12/2011 • Here is another post about how to read news reports about a medical claim and evaluate for yourself if it's valid or not. My last post dealt with case reports -- doctors' descriptions of patients ...more
07/23/2011 • In previous posts I described various kinds of medical evidence, ranked from less to more reliable. This time I'll lay out how researchers collect the best evidence -- something called the randomized, controlled trial. These ...more
- How to read the medical news III: case reports and descriptive series | Christopher Johnson M.D. PICU Author
Leave a Reply
01/29/2013 • Jerome Groopman, Harvard professor and staff writer for The New Yorker, has written a book called How Doctors Think. It's been on various best-seller lists, and deservedly so, since its publication. His point is that, among other ...more
08/17/2012 • This is an excerpt from my latest book. The particular chapter is about ear infections -- what they are and what they would look like if you could see the microscopic action up close. Before ...more
01/31/2012 • No pediatrician I know has ever liked any of the many over-the-counter cough and cold remedies very much, especially for very young children. There never has been any evidence that they help cold symptoms, and what's ...more
06/26/2011 • Whooping cough, or pertussis, its official medical name, is one of those things we hear our grandparents talk about. It once made many, many infants extremely sick with severe coughing and excessive mucous, and some ...more