A Nov. 19 study, of 45,231 women, published in JAMA Pediatrics, identified a heightened risk of autism spectrum disorder (ASD) diagnosis in the children of mothers who received a flu shot during their first trimester of pregnancy. The study, Association Between Influenza Infection and Vaccination During Pregnancy and Risk of Autism Spectrum Disorder, was authored by Ousseny Zerbo and his colleagues affiliated with the Division of Research at Kaiser Permanente.
While the researchers found no increased risk when the mother received flu shots in the second or third trimester, the data demonstrated a 20 percent higher risk of an autism spectrum disorder among children of mothers receiving the flu vaccine during the first trimester. That risk was statistically significant. (The P value, .01, indicates a 99 percent likelihood that the result isn’t due to chance.)
However, after completing this analysis, the authors made a series of adjustments that have drawn criticism from other scientists. Most controversial was their questionable decision to apply a statistical device called the “Bonferroni Correction” to their data. Statisticians use the Bonferroni Correction in very specific circumstances—where they seek to reduce the chance for false positives in calculations involving multiple comparisons. The impact of the Bonferroni Correction is nearly always conservative; it dampens signals in data sets. In doing so it creates the risk of missing true associations. When applied to the first trimester flu vaccine dataset, the Bonferroni Correction reduced the significance of the association from 99 percent to 90 percent. Despite the fact that the adjusted result was still considered marginally statistically significant, the authors then made a second dodgy judgment, by declaring that, “this association could be due to chance.”
These sweeping decisions allowed the authors to arrive at the questionable conclusion that, “There was a suggestion of increased ASD risk among children whose mothers received influenza vaccinations early in pregnancy (first trimester), although the association was insignificant after statistical correction for multiple comparisons.” The researchers summed up with an acknowledgement of the uncertainty of their conclusion: “We believe that additional studies are warranted to further evaluate any potential associations between first-trimester maternal influenza vaccination and autism.”
National media outlets universally missed that nuance. Journalists widely reported the study as a decisive exoneration of flu shots. NPR declared: Flu Shots Don’t Increase Autism Risk In Pregnancy. Fox News celebrated: Flu—or flu vaccine—in pregnancy not tied to autism in kids. The Scientist headlined: Autism Not Linked to Flu or Flu Shot During Pregnancy, while the New York Daily News assured: No link between flu or flu vaccine in pregnancy and autism: study.
As the mainstream media celebrated, public health advocates and scholars cried “foul.” Dr. James Lyons-Weiler, PhD, the CEO and director of the Institute for Pure and Applied Knowledge, and data manager of more than 100 biomedical research studies, told me that the author’s “incorrect” and “unorthodox” application of the Bonferroni Correction in this circumstance risked the appearance that they were using improper methodologies to, “make an unwanted but statistically significant finding vanish in a sea of statistical wizardry.”
Sander Greenland, professor of Statistics and Epidemiology at UCLA’s School of Public Health and College of Letters and Science, agreed that the use of the Bonferroni Adjustment was inappropriate in this context. Greenland is among America’s preeminent statisticians with more than 300 peer reviewed publications—two of which have been cited more than 500 times. He is editor of the Dictionary of Epidemiology.
Greenland said the research team’s use of Bonferroni makes no sense “where there are finely correlated outcomes” and where the cost of a false negative is high—the possibly erroneous conclusion that first trimester flu shots are safe. (See at the end of the post Dr. Greenland’s detailed explanation of the Bonferroni and why it was inappropriate.)
Greenland observes that “in a context like this, it’s something that’s usually called up, after the fact, when they get some significance like this, where they don’t like it and they want to see if they can get rid of it that way. It’s obvious why they used it. It makes the so-called significance go away and, of course, that’s the goal. They’re trying to make things go away…that’s sort of a standard strategy now—by a large segment of the pharmaceutical experts that try to get rid of things. They didn’t like the results and they jumped on it with the Bonferroni. It’s not appropriate here.” Greenland added, sympathetically, that the deception was probably not deliberate, “I don’t think they think this out loud in their minds, it’s just completely Freudian unconscious.”
Dr. Bruce Lanphear, MD, MPH. came to similar conclusions. “Influenza vaccination during the first trimester of pregnancy was associated with about a 20 percent increase risk for ASD.” Lanphear is a physician trained in public health and epidemiology. He is a senior scientist at the Child and Family Research Institute at British Columbia’s Children’s Hospital and a professor at Simon Fraser University’s Faculty of Health Sciences in Vancouver, British Columbia.
His research has focused on the impact of low-level toxic chemicals and metals on brain development. Dr. Lanphear describes himself as a proponent of childhood vaccines. “It is likely that the authors, like me, were biased toward not showing influenza vaccination was associated with ASD,” he said. “That is because many of us in public health are convinced of the benefits of childhood vaccines; that is, we are biased towards vaccines providing benefits. Because we are biased toward vaccinations we might be willing to overlook what appears to be a ‘small association’, maybe even without being conscious that we are doing it.”
Before the authors applied the Bonferroni, Lanphear said, “there was a statistically significant association of influenza vaccination during the first trimester with the development of ASD. If you want to be cautious, or skeptical, and conclude that first trimester influenza vaccination was not associated with the development of ASD, (the Bonferroni) correction allows you to do that. If, on the other hand, you want to be careful not to miss a possible association of influenza vaccination and the development of ASD, you can also reach the conclusion, based on this study, that first trimester influenza vaccination was associated with a 10 percent to 20 percent increase in the risk of ASD.”
Critics of the study suggest that the authors used a number of other devices that may have reduced the autism signal, even before applying the Bonferroni. Dr. Brian Hooker, a biologist at Simpson University in California, said, “the study excluded twins and multiple births, a cohort that has a higher ASD rate” (possibly due to lower birth weight enhancing the risk of a standardized dose). Furthermore, the authors don’t reveal the ages of the children.
“This is another weakness that could have dimmed the signal.” Hooker explained that the median age for ASD diagnosis is about 4 years, 2 months according to the Center for Disease Control and Prevention (CDC) data. If the study failed to follow the subjects to age 8, “It could have missed many potential autism cases.” Also, the study ended in 2015.
“It’s probable that many of the children from the later birth years had not yet been diagnosed. This would also mute the signals. None of these questions are answered in the study,” he said. The study authors have not responded to multiple requests to review their raw data. “Without the raw data, no one can really assess this study,” Dr. Hooker concluded.
Greenland agreed that the authors should not have eliminated multiple births, but is less concerned by these criticisms. “The only thing that raises any eyebrows is the Bonferroni Correction.” Greenland points out that the sample was too small to make decisive associations.
“Their real problem is sample size,” he said. “It’s pretty clear from the numbers that you’re probably not going to be able to discern interactions clearly because the [autism] case numbers are too low. How many cases did you end up with total? That’s what gives you power and precision. Your precision goes up the more cases you have.”
Lanphear pointed out that while financial conflicts of interest among a study’s authors are not fatal, they must be disclosed, and, when they exist, “the results have to be scrutinized more carefully.” The co-author of this study, Nicola Klein, MD, PhD, reported having received previous grant money from a number of vaccine manufacturers, including GlaxoSmithKline, Sanofi Pasteur, Merck, Pfizer, Protein Science, Med Immune and Novartis.
Another co-author, Lisa Croen, does not report any conflicts but her history suggests a potential, unreported, bias—a long time Kaiser employee, Croen was an early autism epidemic denier. Her erroneous calculation in a 2002 study in the Journal of Autism and Developmental Disorders helped fuel the epidemic denial movement. A devastating rebuttal to that study in a published letter by Blaxill et al. 2003, shows how Croen and her co-authors, inappropriately used “diagnostic substitution” to obviate the clear increase in autism cases in California. She was also a co-author on the fatally flawed Price et al. 2010 Pediatrics study, where the authors buried the statistically significant relationship between maternal thimerosal exposure and regressive ASD. The Price study was eviscerated by Desoto and Hitlan (2013) for using the “overmatching” phenomenon to obfuscate the relationship between thimerosal exposure and autism. Croen, Zerbo and other members of the team may also be conflicted due to the nature of Kaiser’s business. Kaiser purchases vaccines in bulk and disseminates them through its clinics and hospitals. If vaccine uptake is low, Kaiser doesn’t get reimbursement. In this respect, Croen’s conflicts of interest are similar to those of a pharmaceutical company researcher. Kaiser has been involved in a scandal in which the company lost critical data supporting a seminal study, which found a link between brain injury and thimerosal.
In 2004, CDC’s Dr. Melinda Wharton informed the Institute of Medicine’s Data Panel that Kaiser had mysteriously “lost” the vaccine data that had revealed a connection between mercury exposure and brain damage. The company “lost” the raw data on the CDC funded Verstraeten study, which linked childhood thimerosal exposure to a 760 percent increase in autism diagnoses. [This loss was highlighted in the IOM Committee’s report Vaccine Safety Research, Data Access and Public Trust, released Feb. 17, 2005.] Kaiser’s credibility as a neutral data manager of the publicly funded Vaccine Safety Database (VSD) has, more recently, come under attack due to the company’s systematically efforts to block access to independent scientists who report adverse reactions to vaccines.
The Study’s Authors Failed to Examine the Role of Thimerosal in Flu Vaccines
Perhaps the Zerbo flu study’s most glaring omission is its failure to separately analyze the effect of thimerosal-containing flu vaccines. Thimerosal is an antibacterial vaccine preservative that is 50 percent ethyl mercury—a potent neurotoxin. According to the CDC, roughly one-third of flu vaccines (48 million out of 160 million this year) contain megadoses of thimerosal. Thimerosal is linked to ASD in more than 55 published studies.
A storm of controversy has battered the CDC since it first recommended, in 2004, thimerosal-containing flu vaccines for women, during pregnancy. The CDC made these recommendations despite a 2001 review of Thimerosal Containing Vaccines and Neurodevelopmental Disorders, where the Institute of Medicine recommended that “full consideration be given by appropriate professional societies and government agencies to removing thimerosal from vaccines administered to infants, children and pregnant women in the Unites States.”
The California EPA has proclaimed thimerosal as a human reproductive toxicant. The agency stated that, “the scientific evidence that thimerosal is a reproductive toxicant is clear and voluminous” and includes “severe mental retardation, or malformations in human offspring who were poisoned when their mothers were exposed to ethyl mercury or thimerosal while pregnant.”
Under the CDC’s flu vaccine recommendations, average-sized pregnant women could be exposed to organic mercury that exceeds the EPA safety guidelines by a factor of 3.5. The fetus could potentially be exposed to a dose of mercury 200 times the EPA guidelines.
The authors of the JAMA Pediatrics article have not responded to our questions about whether they had the data to perform a separate analysis of thimerosal-containing vaccine exposures. Dr. Lanphear speculates that since the authors had the children’s medical records, they probably had the ability to identify the particular vaccine, by lot number, and differentiate the mothers who had received a thimerosal vaccine from those who had not. Dr. Hooker agreed. “The authors apparently elected not to separate out the mothers who received non-thimerosal flu shots during pregnancy from those who received thimerosal-containing flu shots,” he said. “This decision is troubling because, if most of the autism cases were found in children who did receive thimerosal exposure in utero, we would have valuable information about the risk, long suspected, of giving mercury vaccines to pregnant women.”
Is the Risk of Autism Worth the Benefits Received from a Flu Shot?
Zerbo and his co-authors reported that, “If influenza vaccination during the first trimester of pregnancy causes ASD, our results suggest that it would amount to 4 additional ASD cases for every 1,000 women vaccinated—about 4,000 children annually by our calculations.” So,does the flu shot yield net benefits great enough to make up for the 4,000 added cases of autism?
CDC’s rationale for recommending the flu shot for all pregnant women, during any stage of pregnancy, arose from concerns that the growing fetus can reduce the mother’s lung capacity during pregnancy, placing her at an increased risk for pneumonia or other complications if she were to develop the flu. However, research has not supported these concerns. The largest study to date in approximately 50,000 pregnant women over five flu seasons found no difference in the risk for developing influenza illness among those who received the vaccine during pregnancy compared to those who did not. Infants born to mothers who received the vaccine also had the same risk for influenza or pneumonia as infants born to unvaccinated women. The authors reported that “hospitalization admissions for influenza or pneumonia were quite rare and no women died of respiratory illness during pregnancy.”
Neither have Flu Vaccines Been Subjected to Appropriate Pre-Clinical Reproductive Toxicity Testing
CDC’s universal flu vaccine policy for all pregnant women during any trimester of pregnancy is troubling because flu vaccines have never been subjected to appropriate safety testing. In 1998, a Food and Drug Administration working group recommended that “preclinical reproductive toxicity studies should be conducted for every vaccine indicated for immunization of pregnant women.” These toxicity studies have never been performed. The vast majority of flu vaccines administered to pregnant women are labeled as “pregnancy category C” drugs. This is the information provided by the manufacturers of a popular influenza vaccine: “Pregnancy Category C: Animal reproduction studies have not been conducted with Fluzone. It is also not known whether Fluzone can cause fetal harm when administered to a pregnant woman or can affect reproduction capacity. Fluzone should be given to a pregnant woman only if clearly needed.”
Accumulating research indicates that flu vaccines administered during pregnancy can induce an inflammatory response in the mother that can cross the placenta and potentially cause harm to the fetal brain during critical windows of neurodevelopment, including harm associated with autism.
A study by Christian, in 2011, found an increase in two inflammatory markers, C reactive protein (CRP) and tumor necrosis factor-alpha (TNF-α) in pregnant women given a seasonal flu vaccine. Increases in these inflammatory compounds indicate a significant level of inflammation, which was identified during the first two days following vaccination. There is good reason to be alarmed by these findings. A 2014 study by Alan Brown, MD and his colleagues of more than 1.2 million pregnant women found that elevations in CRP, the same marker of inflammation that increases after flu vaccination, are associated with a 43 percent greater risk of having a child with autism. An earlier study by Lisa Christian, PhD and her colleagues found that pregnant women suffering from depression developed a more marked inflammatory response to influenza vaccines than women who did not have symptoms of depression. Researchers found that depression during pregnancy has also been associated with an increased risk of autism.
The CDC’s current recommendation that all pregnant women receive flu shots is a dangerous proposition according to California Institute of Technology Professor Paul Patterson, a neuroscientists whose research has included immune activation during pregnancy. In his research, Dr. Patterson reported that any immune activation during pregnancy, whether from an infection or a vaccine, can cause damage in the developing fetal brain.
In 2008, Scientific American Mind quoted Patterson criticizing the CDC’s policy of administering flu vaccines during pregnancy. “I don’t think they have considered this risk,” he said. “In fact, I know they have not considered this risk.” Patterson elaborated, “If you take it seriously and vaccinate everybody, then what is going to happen? Researchers cannot yet predict how often a prenatal immune response might lead to fetal brain damage, but even if it happens less than 1 percent of the time, vaccinating an entire population of pregnant women could affect thousands of children.”
In a study published in the British Medical Journal in 2014, Giuseppe Traversa and colleagues assessed maternal, fetal and neonatal outcomes of women given the influenza A/H1N1 vaccine. The outcomes of more than 86,000 pregnancies revealed that vaccinated women had significantly higher rates of gestational diabetes and eclampsia. Eclampsia is the development of seizures in a woman with severe toxemia, a condition characterized by high blood pressure and protein loss in the urine. Eclampsia is fatal in 2 percent of women and can result in long-term health problems in those who survive. Fetal complications, including neurological damage and death, are also common. Both gestational diabetes and eclampsia are related to inflammation and immune dysregulation, making the connection to the immune stimulation of the flu vaccine very plausible. Research by Goldman in 2013 documented a 11 fold increase in fetal loss reports following the 2009-2010 pandemic flu season when pregnant women received two flu vaccines during pregnancy, the seasonal flu vaccine and the H1N1 vaccine. A 2016 study by Chambers et al. found a moderately elevated risk for birth defects among children born to mothers who received one flu vaccine during the 2010-2014 flu seasons.
Reduced lung capacity in the mother is only an issue in the last trimester of pregnancy due to the growing size of the baby. Administering flu vaccine during the last trimester of pregnancy and only when that occurs during flu season would be a more precautionary approach given these recent findings. However, the JAMA Pediatrics authors do not call for a change in flu vaccination policy. They only recommend further research. That weak endorsement should concern millions of women who are pregnant, or hoping to become pregnant and wondering whether to receive a flu shot. After all, this is not a decision that mothers should make without careful consideration of the best data available. Recent research, including the Zerbo study, plants real fears that influenza vaccines, while intended to protect pregnant women and their babies, may inadvertently be doing more harm than good.
Here’s Dr. Greenland’s Detailed Explanation of the Bonferroni and Why it was Inappropriate:
I asked Sander Greenland, by email, to explain the Bonferroni Correction in the context of the flu study. Here is what he wrote back:
“Bonferroni is a ‘multiple comparisons adjustment’ for reducing the risk of false-positive findings when engaging in statistical ‘fishing expeditions’ among many unrelated associations. It is appropriate only when any of the following are true:
- those associations are equally important, likely, and expected to be zero (absent) based on external (a priori) considerations;
- the cost of any false negative is minor compared to the cost of any false positive; and
- the associations are independent (unrelated) to one another.
In return for the reduce risk of false positives, multiple comparison adjustments, like Bonferroni, dramatically increase the risk of missing real associations (false negatives). So, even if there were no other objections, Bonferroni as used by the authors (with N = 8) is simply erroneous.
Using Bonferroni in this study was wrong for several other reasons:
First, the authors specifically wanted to test if influenza vaccination during pregnancy was a risk factor for ASD—this was not a ‘fishing expedition” as assumed by Bonferroni (violating ‘a’ above).
Second, the overall association of influenza vaccination anytime during pregnancy depends completely on the associations within each trimester, so violates the Bonferroni assumption of independence (violates =c= above).
Third, the first trimester is expected to be the period of greatest vulnerability for the developing fetus, and so is a pre-specified hypothesis. (In other words, before the study, the stakeholders expected (a priori) an association, which also violates ‘a’)
Finally, we need to be confident that vaccines are safe: the costs of wrongly concluding that the influenza vaccine is safe rivals the costs of wrongly concluding that it causes harm, which violates the Bonferroni assumption (‘b’) that wrongly concluding harm is more costly than wrongly concluding safety.
To sign up for updates from Robert F. Kennedy, Jr., go to the World Mercury Project.