STATS ARTICLES 2009
The BPA controversy: When journalists can’t tell good evidence from bad
Trevor Butterworth, November 6, 2009
Toxicologists overwhelmingly believe that journalists can’t spot bad science. Newsweek took on the challenge with BPA. Here’s how it failed.
When STATS found that an overwhelming majority of toxicologists in both academia and industry – 97 percent – believe that journalists do a poor job of distinguishing good from bad studies, we hoped that Newsweek might give them pause to reconsider the possibility that journalists can figure out why some studies are more reliable than others. After all, that was the focus of an article written by science editor Sharon Begley -- “When Studies Collide; Rethinking the evidence on BPA.”
The Newsweek article contended that if the Food and Drug Administration’s rethink on the safety of BPA finds it dangerous, the plastics industry will trot out the phrases “Research shows,” “Studies have found,” “Scientists conclude”….. all with the ending "that BPA is perfectly safe.” It continued:
“Whether that's true can be answered only by empirical data. But not all empirical data are created equal. BPA studies that a spokesman for the American Chemistry Council described as "definitive," for instance, have come in for criticism on three fundamental grounds, not including that they were partly funded by industry (I don't reflexively assume that industry-sponsored research is suspect; whether a study is good or not depends on how it was conducted). First, research in 2002 used a strain of rat that is extremely insensitive to estrogen; it doesn't even show hormonal effects if it's given 100 times the dose of estrogen in human birth-control pills. Since BPA acts like an estrogen, finding no effect in this insensitive rat is about as illuminating as not finding an effect of rain on a waterproof watch. That doesn't tell you that water can't harm machinery. Second, a 2008 study found that prostates in mice not exposed to BPA--these were the control animals--were 70 percent larger than normal. That's a problem: other studies have shown that BPA enlarges the prostate by about 35 percent. If you're looking for a prostate effect by comparing BPA-exposed mice to mice with mysteriously abnormal prostates, it's no wonder BPA gets exonerated. Finally, another 2008 study compared BPA to estradiol, a form of estrogen. But estradiol had never been used to provide such a baseline, so concluding that BPA is less potent than estradiol--as industry does--is like saying one temperature is higher than another when you don't even know if the thermometer works.”
But none of these criticisms stands up to scrutiny.
First, the Center for the Evaluation of Risks to Human Reproduction, part of the National Toxicology Program, does not agree with Newsweek that the Charles River Sprague-Dawley (SD) rat is insensitive to estrogens and should be discounted in BPA research. The CERHR notes that there are even studies showing low dose effects from BPA in SD rats.
“The sensitivity to estrogens has been mapped to specific chromosomes for several traits. In no case has it been demonstrated that the SD strain is completely insensitive to any known estrogen. It is evident that different traits map to different chromosomes and the degree of estrogen sensitivity varies from tissue to tissue, likely depending on the tissue-specific gene regulated by ER on the chromosome.”
Where does Newsweek’s evidence that the National Toxicology Program’s CEHRH (composed of some of the nation’s top experts on reproductive health and toxicology) is wrong come from? The article doesn’t say.
But more to the point – the key study behind the European Union’s decision that BPA was safe was *not* this unnamed 2002 study. It was a 2006 study the European Food Safety Authority supervised (and had industry pay for) according to international protocols for ensuring methodological rigor and statistical reliability. This two-generation reproductive toxicity study controlled for animal strain, feed, and the use of positive controls, and while the control, estradiol, produced effects, BPA didn’t. (Tyl et al, Two-Generation Reproductive Toxicity Study of Dietary Bisphenol A in CD-1 (Swiss) Mice, Toxicological Sciences 2008 104(2):362-384).
Second, what about estradiol as a control? Newsweek is incorrect in stating that estradiol (not so much a “form of estrogen” but the major estrogen in humans) has never been used as baseline. No one is simply saying, “hey, estradiol is more potent than BPA” – so are soy milk and the phytoestrogens from vegetables; rather, to determine whether mild estrogenic compounds can have estrogenic effect at very low doses, one needs to use a control substance – a known and more powerful estrogen at the same range of doses for comparison and to establish a benchmark. Estradiol was used as a control in reproductive toxicity studies as far back as 1998 (Biegel et al, Effects of 17ß-Estradiol on Serum Hormone Concentrations and Estrous Cycle in Female Crl:CD BR Rats: Effects on Parental and First Generation Rats, Toxicol. Sci. (1998b) 44:116–142.) – because it can provide proof that the thermometer is working.
Crucially, estradiol was also used as a control by Kembra Howdeshell in a major 2007 study by the EPA not mentioned by Newsweek. Howdeshell found that lactational and gestational exposure to both estradiol and BPA over a broad range of orally-administered low dose endpoints in rats only produces effects for estradiol. (Howdeshell et al, Gestational and Lactational Exposure To Ethinyl estradiol, But Not Bisphenol A, Decreases Androgen-dependent Reproductive Organ Weights and Epididymal Sperm Abundance In The Male Long Evans Hooded Rat, Toxicological Sciences, Toxicol. Sci. 2008 102: 371-382; doi:10.1093/toxsci/kfm306). A second, more recent study by the EPA also failed to find effects for BPA. Both studies used a different variety of rat to boot.
The other problem with denying that estradiol can be a control is that all these studies were accepted for peer-reviewed publication; indeed, the Tyl and EPA studies were published in Toxicological Sciences, the most prestigious of the toxicology journals. If Newsweek is right, the editors and reviewers for these publications don’t understand basic toxicology.
Third, as for the case of mysteriously large prostates in mice in Tyl et al (2008), the author addressed this issue in a letter to the FDA. She noted that prostate weights varied depending on the age of the mice, and that when compared to other BPA studies that measured prostate weights, her findings were well within the overall range of weights in other published studies.
Tyl expanded on this and other criticisms in a subsequent paper that also went ignored in by the media: Basic Exploratory Research Versus Guideline-Compliant Studies Used for Hazard Evaluation and Risk Assessment: Bisphenol A as a Case Study (Environmental Health Perspectives Volume 117, Number 11, November 2009)
She noted that her lab had been rated among the best labs in the world for testing prostate weights. She also noted the source of the prostate and the other criticisms mentioned by Newsweek: an article co-written by Frederick vom Saal, the controversial scientist who began campaigning against BPA over a decade ago and whose work has been repeatedly rejected by international and independent risk assessments.
Among some of the interesting rebuttals raised in Tyl’s article, which includes an extensive list of “BPA is dangerous” studies that couldn’t be replicated, was that her “mouse BPA study was also under formal oversight by noted reproductive toxicologists from the United Kingdom, Denmark, Germany, the Netherlands, Sweden, and the European Chemicals Bureau.”
Newsweek also failed to note the key methodological difference distinguishing the good from the not so good BPA research, which is whether the study injected or fed the chemical to its subjects. The route of exposure has a different impact on the body. Oral exposure eliminates BPA’s estrogenic capacity in humans, as noted by the European Food Safety Authority. AS we get our exposure to BPA orally, studies conducted by injecting the chemical are not relevant to risk assessment. Tyl notes that the studies which found adverse effects from BPA exposure all relied on subcutaneous pumps or implants, which international bodies have rejected as being INappropriate [!!!] for evaluating human exposure and risk assessment.
Vom Saal and others have argued that injection is validated by the fact of grant approval for their research by the National Institutes of Health. But the National Institute of Environmental Health Sciences, which funded a number of these injection studies (for asuitable hysteria-inducing example click here), has now decided that in further research on BPA the “route of exposure should be oral or justified to provide similar blood levels as the oral route.” In other words, the NIEHS now endorses the methodology used by Tyl.
Tyl provided one of the key studies used by the European Union for assessing the risk from BPA. And since that risk assessment in 2006, there has been a review by Japan’s National Institute of Advanced Industrial Science and Technology (2007); an examination of claims of neurotoxicity by the Norwegian Scientific Committee for Food Safety (2008); an update to the European Union’s risk assessment (2008); an evaluation by the French Food Safety Agency (2008); a risk assessment by NSF International, a World Health Organization collaborative center (2008); a review of new data by the German Federal Institute for Risk Assessment (2008); a joint regulatory review for manufacturers by the FDA and Health Canada; a survey by Health Canada (2009); a risk assessment by Food Standards Australia/New Zealand (2009); two more surveys by Health Canada, one on canned powdered infant formula, the second on bottled water products (2009); a hazard assessment by California’s Environmental Protection Agency (2009); and a modeling study of BPA in humans by the German Federal Institute for Risk Assessment (2009).
All have rejected the evidence that Newsweek seems to find “both stronger and more convincing.” All have found BPA, in contrast, to be safe. Even Health Canada, which banned BPA in baby bottles, admitted that it found no hard evidence that anyone was at risk, thereby inviting ridicule from France’s Health Minister, Roselyne Bachelot - “Canadian authorities banned bisphenol A under public pressure and without any serious scientific study”
All of which, sadly, brings us to another finding from our survey of almost 1,000 members of the Society of Toxicology, the professional organization for scientists studying the risks of chemicals: When it comes to the national news media’s track record of accurately describing chemical risks, you’d be better off going to Wikipedia.