STATS ARTICLES 2006
Another Crazy Columbia Alcohol Study
May 08, 2006
Rebecca Goldin Ph.D
Columbia University research riddled with substantial errors; wildly overestimates underage drinking, abuse, benefit to drinks industry
Errata appended January 15, 2007
May saw the appearance of more academic research claiming that the alcohol industry profits from alcohol abuse and dependence, and from underage drinking. Most news organizations, perhaps mindful that the source of this data has a long track record of exaggeration and error, gave the study a pass, but The New York Times and Forbes were hooked into reporting that almost 50 billion dollars per year of revenue results from alcohol abuse and dependence, and minors consuming alcohol. Previous CASA estimates were even higher, the articles point out, amounting to almost half of the industry's revenue.
The source of these claims is a recent study published in the Archives of Pediatric and Adolescent Medicine, which attempts to analyze the number of drinks per year consumed by underage drinkers, the number of drinks per year consumed by those 21 and over who abuse or are dependent on alcohol, and the cash value of this alcohol. The study also examines the percentage of underage drinkers whose use can be classified as abusive or dependent. The main author of the study, Susan Foster, works at the National Center on Addiction and Substance Abuse at Columbia University (CASA), an alcohol abuse think-tank that is notorious for its overstatements of abuse and its consequences.
Without dismissing the substantial cost (and profit) associated with abusive or illegal drinking, we looked into the validity of the claims, and we found the study riddled with errors. Unfortunately for those serious about research on alcohol use, the peer review process did not challenge substantial discrepancies in the numbers. Unfortunately for the public, the media didn’t look into whether the findings made sense, or whether there was concurrence about these ideas in the community of researchers looking into underage drinking, abuse, and dependence.
The New York Times titled their piece “Addiction: Sales Estimates Paint Portraits of Alcohol Abusers” and quoted the authors of the study without validating any figures. Neither did the paper ask other alcohol abuse experts if the findings were justifiable. In addition, the Times reported that “The study said the drinking behavior of a quarter of the underage drinkers surveyed already could be considered alcohol abuse.” This conclusion is swallowed and cited, though the CASA authors are statisticians and not psychologists, and the study is not designed to determine abuse levels.
Here are a few numbers that don’t make sense: according to their estimate, over 20 billion drinks are consumed by underage drinkers. STATS was unable to reconstruct this number. According to their own analysis, 47.1 percent of kids age 12-20 are “drinkers”, that is, they consume at least one drink per month. According to the 2000 U.S. Census, there are 35.8 million people in the United States in this age range; of these, just under 17 million drank in the past month. The average number of drinks/month, according to the data given in this article, is 35.2 per person per month– or about 422 per year. This amounts to about 422 times 17 million, or just under 7.2 billion drinks per year, far from the 20 billion reported in their table, and used for their analysis. For these same kids to consume 20 billion drinks, each teen would have to consume over 1,000 drinks per year, or almost three drinks a day!
Even more confusing is that fact that in this article the author claims that drinkers age 12-20 average 35.2 drinks a month. But a previous article by the same authors claims to have obtained an average of over 46 drinks per month, using the same data – though STATS couldn’t reconstruct that 46 a month either (we got 34.6 based on their reported averages).
We also took a closer look at how the authors determined the number of drinks each person averages in a month. They use data obtained by the National Household Survey of Drug Abuse (NHSDA), which surveyed kids in this age range about their drinking habits when their parents were home. This same data had been dismissed by Foster as being unreliable for finding the percentage of kids who drink, since they are likely to underreport their drinking, for fear of getting in trouble.
Instead, Foster uses data from Youth Risk Behavior Survey (YRBS) to calculate the percentage of youth age 12-20 who drink. But Foster accepted the NHDSA data reporting how much someone who does admit to drinking actually drinks. They then multiply this drinking rate by the proportion of kids 12-20 who admit to drinking in the YRBS survey. In other words, Foster assumes that the population of drinkers age 12-20 drink the same quantity as the population of drinkers willing to admit it.
This maneuver is likely to have produced drastically skewed results. Let us suppose that kids who drink some – but not much – are more worried about what their parents think (or hear during the NHSDA phone interview) than heavier drinkers. Those who drink a lot would report their average drinking rates, but those who drink a little would simply say they don’t drink at all. The average number of drinks per month would then reflect averages for the heavier drinkers, and would make it look as if 12-20-year olds are drinking more than they are.
While we can't know what wasn't reported on a survey, it stands to reason that a 12-year-old drinker who drinks a beer once a week is less likely to report that he drinks at all than an 18-year-old drinker who binges every other day. If lighter drinkers are hiding it (because they're younger or overall more worried about what their parents think) than heavier drinkers, this could have a drastic effect. But Foster's over-simplistic statistical model assumes just that.
There are other problems of exaggeration in this study as well. According to their own analysis (based on data not shared in the article), some 25.9 percent of underage drinkers are dependent or abuse alcohol. But this “diagnosis” was administered by the people conducting this study – not by independent evaluations – and was based on questions asked on a survey (again, using the unreliable NHSDA data).
The DSM-IV criteria for abuse, for example, require someone to experience one of four conditions in the past year: role impairment (failed work or home obligations), hazardous use (e.g. driving while drunk), legal problems related to alcohol use, or social problems due to alcohol use. Several of these measures are subjective, and not easily recognized on a questionnaire; furthermore, since drinking is illegal for people under 21, the rate of “legal problems” obviously goes up for this age group, whether the drinking is abusive or not. The net result is that a larger percentage of kids will be classified as “abusing” alcohol by this method than the same behavioral patterns would elicit if they were over 21.
And yet again, the proportion of thos who misuse alcohol or are dependent on it among drinkers is determined by NHSDA data, but the proportion of drinkers in the whole population is determined by other data. So if abusive and dependent drinkers are more likely to talk about their drinking habits during an NHSDA interview than those who drink lightly, the analysis will suggest that underage drinkers are more abusive/dependent than they in fact are. And sure enough, Foster’s result is that almost 26 percent of youth drinkers are abusers or dependents, compared to under ten percent of adult drinkers.
Here’s the financial rub. After assuming that kids are as likely to buy expensive spirits as adults, they also assume that the kids buy as-expensive brands of spirits as adults do – an admitted weakness of the study, which they have no way to account for (and so they don’t). This inflates the “cost per drink” that is then valued for the analysis, augmenting the benefit for the industry.
Finally, the study attributes all drinks consumed by people who misuse or are dependent as alcohol that shouldn’t have been sold. In other words, any revenue obtained from people who abuse alcohol is revenue reported as income in this category, not just drinks that are misused. But someone who satisfies one of the four criteria for alcohol abuse – such as driving when drunk once over a year, or having trouble at work due to alcohol once in a year – may consume many, many drinks over a year in a non-abusive situation. Few would argue that the industry should not have profited from the many "responsibly consumed" drinks by a person who is classified as an alcohol abuser.
The upshot of all this: the number of drinks consumed by youth under 21 is overestimated, the cost per drink is overestimated, the amount of drink attributed to abuse and dependence is overestimated, and the benefit to the industry of youth drinking and alcohol abuse and dependence is overestimated.
Even with more careful analysis of the economics of alcohol consumption, the financial impact of underage and abusive/dependent drinking is significant. There is no need to inflate the figures. An eyeball look at this study suggests that the benefit to the industry was overestimated by as much as a factor of five. But even a benefit of 10 (instead of 50) million per year is impressive.
It is highly dispiriting to find scientists at an Ivy League university spiking their research to give it extra punch. It is even more troubling to find the peer-review process so deficient in catching these mistakes. The authors of this study claim, as reported by Forbes that, "The financial interests of the alcohol industry appear to be antithetic to the public health interests of the nation in preventing and limiting pathological drinking." We would argue that the public health interests of the nation are not being served by an organization that is so lacking in scientific rigor – or by the news organizations that report these studies without any critical reflection.