How to use medical evidence V: population studies

Here is another post in understanding the proper use of medical evidence. Medical researchers can use several techniques to try and get around some the pitfalls you read about in earlier posts. Various kinds of population studies are examples of how they do this.

These kinds of studies can be retrospective, meaning we look back in time at things that have already happened to people, or prospective, meaning we follow forward, in real time, a group of patients to observe what happens. They are most often used to determine if there is any association between something, say a drug or an environmental exposure, and a disease.

Finding such an association doesn’t prove causation. We must beware of another logical fallacy, cum hoc ergo propter hoc, or “with this, therefore because of this” in the Latin. But a very strong association between two things is reasonable evidence for a link of some sort between them, such as one thing at least partially causing the other, or else both being caused by some third thing. After all, where there’s smoke, there’s usually fire.

One common tool of this sort of research is the case-control study. In this technique the researcher tries to identify a group of people who have the thing she’s studying (the case group) and match it with another group of people who are just like the case group in such things as age, sex, and whatever else might confound the analysis, but who don’t have the disease (the control group).

The two groups can then be compared, looking for things that the case group has that the control group doesn’t; such things could then be associated with the disease, and maybe even cause it. The crucial part of this process is choosing a good control group. Sometimes each individual case has its own individual control, matched to it as closely as possible. A major problem with case-control studies is that investigators can only match the two groups for confounding variables they know or think at the time may be important for the disease they are studying; it may turn out later some completely different thing was important, something they didn’t control for when they matched the two groups because they didn’t know it mattered.

Here is a simple, hypothetical example of what I mean. Suppose an investigator has a theory a certain food is protective against cancer of the large intestine (colon). She identifies persons who developed the cancer, matches them with others who don’t have the cancer, and then does a detailed analysis of the diets of the two groups to see if there were differences between cases and controls in how much they ate of whatever the theoretically protective food item is. If she doesn’t match the case and control groups for the variable of having a family history of colon cancer, she will be seriously misled by the results because colon cancer is one of those malignancies that tend to run in families.

Case-control studies, if they use interview techniques, can be quite susceptible to recall bias. For example, imagine you are trying to determine if exposure to a particular food is associated with a medical problem. You assemble a group of patients who have a disease and ask them about exposure to whatever food you are investigating. Then you ask the same questions of those in the control group. The trouble is, usually researchers are investigating a potential connection like this because it’s already believed at least by some people, perhaps many people, that there is one. If it’s already widely believed there is such an association, unless you design your survey very carefully, the patients in the case group are more likely to recall an exposure to whatever the food is than are your controls.

Another way to examine, say, the theory a particular environmental agent or activity is associated with developing a disease is to do a prospective cohort study, selecting two groups of people who differ only in their exposure to the thing being studied, and then follow them over time to see what happens. This kind of study is much less likely to be plagued with recall bias because you are not asking people to remember what already happened to them. (It is also more time-consuming, often taking years.) If the hypothesis is correct, then those exposed to whatever the agent or activity you’re studying should be more likely to get the disease than are those who aren’t exposed. Like case-control studies, however, the key is in the selection of the control group — it must not differ from the case group in any way that could affect susceptibility to getting the disease.

Properly done, these kinds of population studies give much more reliable data than do the techniques I discussed in previous posts — expert opinion, case reports, and uncontrolled trials. However, they don’t lend themselves well to treatment trials, assessing what works, because the population researcher is generally just passively observing things. The best way to determine if a treatment works is to use the gold standard, known as the randomized, controlled trial. You will read about how those work in a later post.