How to read the medical news II: expert opinion

This is another post about how non-physicians can understand how physicians use evidence. As I noted before, medical evidence has a hierarchy of reliability. The least reliable of these is expert opinion. This seems counter-intuitive: why is expert opinion the worst sort of evidence? Should not the experts know what they are talking about? In general, of course, experts do know what they are talking about — that is what makes them experts. But a closer examination of the matter shows why this kind of evidence is the weakest and most subjective; after all, it is one person’s opinion (or sometimes a committee of persons’ opinions), and opinions can be incorrect.

We need to look closely at just why a particular individual is considered an expert, and by whom. Credentials are important: where did the person obtain her training, where does she work now, and what is her standing among her peers? These seem obvious questions to ask, but these days an astonishing number of people with dubious or no credentials can write a book, put up a website, or start a blog, and, if they are persuasive marketers, can convince others they are experts. Fortunately for parents, the same wide-open quality of the internet allows one to search the background, credentials, and accomplishments of any putative expert. Wise parents will do this as a matter of course before deciding whether to take the expert’s advice on any important matter about their child’s health.

Experts who advise you to do one thing or another with your child typically base their advice to you on their own interpretation of the available medical research. They have the knowledge and training to understand the often esoteric medical literature. In addition, most experts themselves do research in the relevant field. Those are important and useful things. However, there are still good reasons why we should regard such opinions as the worst kind of data — better than nothing, but sometimes only barely so.

For one thing, the reason an expert holds a particular opinion may be because she was taught that way by her teachers, who may have, in turn, been taught the same thing by their teachers. Medicine is practical and empirical enough that such received, traditional opinions will not be tossed out unless they are consistently wrong. It is also true we physicians venerate our medical forebears to the extent that misguided opinions can occasionally persist long after they should have been discarded. So sometimes the answer to why doctors do something a particular way is that we have always done it that way.

Experts also form their opinions based upon what they have seen in the past. If their experience is long, often they have seen quite a few instances of whatever is under discussion, and that experience should count for something. On the other hand, memory is a tricky thing; sometimes we recall things in ways that can ultimately prove misleading. For example, the more striking and dramatic things tend to stay in our minds better than the more mundane things, and medical experts are not immune to this phenomenon. For example, I know that I remember unusual manifestations of certain cases for decades, and this inevitably colors how I approach the next child with that particular problem. Even though I know the case was unusual, I naturally think of its circumstances whenever I care for another child with whatever the disorder was. This is an example of what we call recall bias.

There are other kinds of bias that may affect the judgment of medical experts, and some of them are not innocent things like tricks of memory. Medical experts are no different from other kinds of experts, such as foreign policy pundits or stock market analysts, in that we, too, may have agendas that are not obvious to parents listening to our advice. Any controversial subject will lead to partisanship, and medical debates are no different. Parents considering the advice of medical experts should be alert to what a particular expert’s agenda might be. This is not necessarily a sinister thing; I think the great majority of experts advocating one position or another do so because they truly believe it is the correct one. But it is still a real thing.

Sometimes, however, medical experts may have agendas that are not so innocent. For example, there have been recent examples of experts touting one treatment over another when they have an undisclosed interest in the outcome. The conflict of interest could be intellectual, such as past friendships or associations with researchers of a particular treatment, or they could be crassly commercial. The ethical boundaries are, in theory, quite sharp, but there have been well-known examples of medical experts recommending a particular treatment when they have a financial conflict of interest regarding the outcome.

The best experts, of course, are the ones who tell you the basis of their opinions. They do not just pontificate about what is best for your child — they tell you how their reading of the best evidence led them to their informed opinion. They interpret for you the meaning of the scientific evidence. On the whole, most medical experts will do this for parents. Even so, it is a good thing for parents not to take whatever the experts say entirely on faith; it is better to have some grasp of how medical evidence is collected and analyzed. And besides, even experts can be wrong.