When you see a news report about new research, it’s important to determine whether the research was experimental or observational.

If you don’t see the words “randomization,” “controlled” or “experimental,” it’s probably an observational study

Observational research can never show that one thing causes another; it can only show that there is a “correlation,” or association, between two or more things.

If experimental studies exist, they are almost always better evidence than observational data and should be given greater weight in your decisions about changing your behavior

Experimental studies can produce misleading results if they only draw on a few subjects.
The more subjects there are in the study, the less likely the results are due to chance or error.

Only “experimental” or “controlled” studies can prove whether a particular drug or activity is the cause or the cure of a particular illness. “Observational” or “epidemiological” studies may find an association or a correlation between taking a drug and an outcome, but they cannot say with absolute certainty that one is the cause of the other.

Consider the debate over hormone replacement therapy (HRT). Until recently, all the evidence in favor of HRT came from observational studies. These are studies that look at large groups of people, and in this case they compared the health of women who happened to be taking hormones with the health of those who were not.

But observational studies were not able to say whether the hormones were responsible for the differences between the groups, or whether it was some other factor or factors common to the group taking hormones and absent from the group that abstained. The group that took HRT may have done so because they were far more health conscious to begin with, or had access to better medical care.

Experimental studies, on the other hand, minimize the impact of pre-existing differences between participants by using random selection (randomization) to determine who takes the drug and who takes a placebo. It was only when experimental studies were done on HRT that researchers discovered a slightly raised risk of heart disease, dementia or stroke (at least for the currently used combination of estrogen and progesterone).

Experimental studies are almost always better than observational ones at reducing the likelihood of chance connections, but observational studies of thousands of people — even though they can’t prove causality — may provide better evidence than tiny experimental ones with less than 12 subjects.

Unfortunately, the media and advocates for various organizations tend to gloss over these critical distinctions for various reasons, from punching up a story to make it seem more exciting, to attempts to persuade people that their cause is valid.

risk home

STATS home