STATS at George Mason University how to evaluate health risks
checklist
 
6. Anecdote is not evidence

“Anecdotes, no matter how compelling, are not ever enough to establish a causal relationship.” - George Gray, Former Acting Director of the Harvard Center for Risk Analysis

 

Physicians, like anyone else, can have pet theories and ideas about what works and what doesn’t; and they may base them on “clinical observations,” which makes them sound more scientific than some story your aunt told about how your cousin got better when he used a herbal remedy.

But clinical observations — like other anecdotes about health — are subject to all manner of biases. For example, research has shown that even trained professionals typically recall cases that support their theories, but forget about or arbitrarily dismiss the ones that don’t.

Doctors can also forget to consider that symptoms ordinarily wax and wane — so what may look like an effective treatment in a particular case might just be one that, coincidentally, was administered when the disease was improving.

If the new “research” being reported is just a series of accounts by a doctor about something she is seeing in her practice, and it has not been published or has only been published as a “case report,” then this is not sufficient evidence to lead you to change your behavior. If the only evidence in favor of a particular treatment is a collection of stories from patients who claim that it helped, or statistics from the treatment’s developer which haven’t been independently verified or published, then don’t panic. Neither can prove anything about the treatment.

Remember, patient anecdotes don’t tell us anything about what would have happened if they received no treatment or different treatment; unverified statistics are subject to numerous deliberate and unintentional errors that just happen to back the result the promoter wants to highlight.

 

 

risk home

STATS home

email this page to a friend

next risk topic

   
 
 
about staff publications get e-mail updates contact us search contribute