STATS ARTICLES 2011
Scary or spurious? Diet soda and the statistical probability of stroke
Rebecca Goldin, PhD and Cindy Merrick, March 3, 2011
What if you drink diet soda to compensate for other unhealthy food?
Drinking diet soda every day is associated with a dramatic increase in your chance of stroke or heart attack, according to the results of new research presented at the recent American Stroke Association’s International Stroke Conference.
The study, which has so far been published only in abstract form, used self-reported data from a food frequency questionnaire given during the Northern Manhattan Study, whose multi-ethnic sample of participants had a mean age of 69. It compared what they said about what they ate against the the total number of vascular events that occurred during nine years or more of follow up.. The study claims the association between diet soda and heart attack persists even when the population is controlled for age, sex, smoking, daily caloric intake, and many other vascular risk factors, like previous cardiac disease.
It is important to note that these results are preliminary, and the study hasn’t yet appeared in a peer-reviewed publication. But this didn’t stop US News and World Report from charging that , “Researchers find a 61% increased risk among those who drink daily.” The story, via the news service HealthDay, quotes a disinterested expert observing the validity of the scientific method, but fails to point to the very real possibility that the results are spurious: When a lot of data is collected, there are bound to be correlations in the data that are not actually present in the whole population.
Many other news outlets, like the Los Angeles Times and ABC News, while not recognizing the preliminary nature of the announcement, have given more measured perspectives by reporting on the widespread criticism of the study, and cautioning against drawing causal conclusions.
Limitations in the study, which were granted by the presenter and author, Hannah Gardener of the University of Miami, include the fact that data regarding key food consumption behaviors was gathered at a single time point. This can be problematic in any study by assuming that consumption behaviors over a period of time are accurately reported on a single occasion. Also, the correlation between subsequent health effects and the reported behaviors could be based on the assumption that the behavior patterns remained the same after being reported.
Another weakness in the study is that no classification of soft drink type was made. News reports quote Gardener calling for further study to confirm the results, but neglect to point out the small sample size: only 116 daily soda drinkers were include (out of the original 2,564 participants). Christopher Cannon, of Brigham and Women’s Hospital told MedPage Today that “this [small sample size] suggests that the finding may be spurious and needs confirmation in a much larger group.” In other words, instead of searching large data sets for possible correlations, researchers in this area should collect data on dietary patterns with the express purpose of seeing whether such a correlation exists. This greatly reduces the chance that any particular correlation was found by chance.
Another interesting point regarding any assumption of a causal relationship between diet soda and vascular events was made by Howard Weintraub of NYU’s Center for Prevention of Cardiovascular Disease. In an email to MedPage Today he wrote, “Unfortunately, it may be that individuals with poor dietary habits do resort [to] some kind of calorie balancing and continue to eat high-calorie sweet foods but reduce their ‘guilt’ by drinking diet soda.” In other words, Gardener could also be seeing the effect of sipping a Diet Coke along with the Big Mac and Ho Hos for lunch.