Is Your Gas Stove Giving Your Kids Asthma?

A new article purports to show gas stoves cause 12.7% of childhood asthma. Is it true or is it bullshit?

The debate surrounding gas stoves seems to have come from nowhere. Accusations that the Consumer Product Safety Commision (CPSC) will try to enact a ban seem to have originated from an announcement that they will be gathering information regarding how gas stoves contribute to indoor air pollution. This was seized on by partisans both in support and against. But what precipitated this renewed interest? It appears to be an article titled “Population Attributable Fraction of Gas Stoves and Childhood Asthma in the United States”. You can access the article here.

What Does the Article Say?

The authors report on an update to a previously published meta-analysis (I’ll explain what that is shortly). They claim that 12.7% of childhood asthma is attributable to indoor gas stove use. The achieved this number by calculating what is called the Population Attributable Fraction. This is an epidemiologic calculation that attempts to determine what percent of cases are likely to have been caused by a certain exposure. This is a useful technique as long as the input variables are valid. That is where the study authors went wrong.

They start by citing their original meta-analysis and reporting the process for potentially updating those results. They report finding 27 potential new studies but did not include any of them because “none reported new associations between gas stove use and childhood asthma specifically in North America or Europe”. To me, this could mean one of two things. 1) The studies were not applicable to their question either because they looked at different outcomes, were in different geographic areas, were duplicate reviews, or some other reason. 2) The studies showed no effect. They do not report why each study was excluded and leave the reader to wonder which one is true.

What is a Meta Analysis

Before explaining to you why I think their methods are junk, I need to explain what a meta-analysis is. In basic terms, a meta-analysis is a process by which you systematically collect the results of previously reported studies and group those results into a new analysis. This is done because many studies are too small to detect differences between groups. The meta-analysis attempts to fix the sample size problem but in the process creates its own issues.

The first mistake that can be made is in selecting which studies to include. The old saying “garbage in, garbage out” applies here. Putting the results of several poorly designed studies together into a meta-analysis does not magically give you valid results. Study selection is also an area where author bias can have a huge effect. The authors decide which studies go into the analysis and are at risk of excluding studies that disagree with their pre-conceived ideas. Additionally, publication bias can result in studies that show no effect being less likely to get published. These negative results could mitigate any positive effects shown in published articles but are not included because they are not known to the authors.

Once the studies are identified for inclusion, a process called weighting must occur. Higher quality studies, such as randomized trials, are often given greater weight in the final analysis than lower quality studies. For example, a well-designed randomized trial that included 100 people might be given equal or greater weight than a 900-person observational study. There are statistical methods for how weighting is performed but, once again, there is some subjectivity in how this is done which provides another way that bias can affect the result.

Once the analysis is complete, the end result is often reported as an odds ratio. This is exactly what it sounds like, a ratio of odds. For this articles example (using made up numbers) let’s say that the odds of a kid in a home with a gas stove developing asthma is 1/5 while the odds in a home without a gas stove is 1/10. This would give an odds ratio of 2. This is sometimes misstated as the kids in the home with the stove being twice as likely to develop asthma. The stats nerds will know why that statement isn’t exactly correct but for our purposes, its close enough. One thing missing from my calculation is a confidence interval. This expresses the uncertainty surrounding a result. If the confidence interval crosses 1, it means that statistically speaking ther is no effect. Now let’s get back to the study.

The Garbage In

The authors note that no new data went into this analysis than what was included in their paper from 2013. I find it hard to believe that no research on this has been performed since that time, but even taking them at their word, the analysis is trash. They discuss their prior results, and this sentence caught my eye: “The combined effect size was based on a North America specific effect size (Nstudies = 3; OR = 1.36, 95% CI = 0.76–2.43) and Europe-specific effect size (Nstudies = 7; OR = 1.34, 95% CI = 1.13–1.60), as reported in a previously published meta-analysis. Notice the North Americal confidence interval clearly crosses 1, indicating no association by statistical convention. No fear, if you combine it with the European studies which did have a significant result, the problem is solved. Lies, damn lies, and statistics strike again.

Let me explain why I think this is absurd. The first is a matter of ventilation and the age of housing. It is logical to assume that newer houses built with gas stoves would have better ventilation, including dedicated ventilation hoods, than houses build more than a century ago. It is also logical to assume that Europe would have a greater proportion of older houses with poorer ventilation, which could easily explain the different results between the US and Europe. This variability (called heterogeneity in statistical terms) means that the inputs may not be similar enough to produce a valid result.

Now that the authors have sufficiently washed their results to get a statistically significant result (studies = 10; Odds Ratio (OR) = 1.34, 95% Confidence Interval (CI) = 1.12–1.57), they use a household survey from the US to evaluate the proportion of homes with gas stoves. Remember, the input was Europe and North America, but the result is only being applied to a survey of US households. The authors make no attempt to justify or explain this decision or why they think their results remain valid and they don’t bring this up in the limitations section. From this point on, however, the methods appear valid (although I am not a PhD biostatistician) and they get the end result of 12.7% (95% CI = 6.3–19.3%).

Biologic Plausibility

Now that I’ve explained why I think their methods are trash, let me throw in this caveat. There is biologic plausibility to their claim. Even if the numbers are wrong, indoor pollutants could put kids at risk of childhood asthma. The authors highlight Nitrogen Dioxide (NO2) as a potential culprit. Nitrogen dioxide causes inflammation in the lungs and constriction of the airways (bronchoconstriction) and in high concentrations it is responsible for a disease called “silo filler’s lung”. This is a disease where inflammation caused by exposure to high concentrations of nitrogen dioxide leads to fluid buildup in the lungs hours to days after exposure. Asthma is a process marked by inflammation, bronchoconstriction, and increased lung secretions that prevent proper air movement in and out of the lungs all of which can be caused or worsened by nitrogen dioxide exposure.

The main method of prevention is proper ventilation. If you have kids or live with someone with asthma, always use the ventilation hood when using a gas range. This will also help decrease exposures to smoke and off gassing from teflon coated pots and pans. With proper care, risk can be greatly mediated.

Final Thoughts

I think I’ve made it clear that this study is not compelling to me despite there being biologic plausibility. There are ways that they could have at least attempted to show that their results were correct. The easiest would have been to take the states that they have the best data regarding percent of households with gas stoves and see if there is a correlation between that at the incidence of childhood asthma. This would not show causation but would bolster their case.

In the end, this is a classic case of epidemiologic data being misused to claim causation, using statistics as a vehicle. The media loves the headlines that this type of analysis provides. I’m left wondering, why now? It has been a decade since the original meta-analysis was performed, and no one seems to have paid much attention to it. It is possible (maybe even likely) that the authors simply wanted to update their analysis but did not find any new studies to include so they decided to calculate a PAF so that they had something to submit for publication. I can’t impute motive, so I’m left with conjecture in that realm.

What I find more nefarious is the way it was rapidly picked up by politicians and media personalities. This is a short article published in a relatively obscure journal and yet it has entered the national conversation. This only occurs when the result is particularly impactful or when it supports a narrative. I’ll leave it to the reader to determine which they think is true.

As always, these posts are not to be construed as medical advice. Everything expressed here is my own opinion. If you notice a mistake, please let me know so that it can be corrected. If you have a topic that you would like me to write about, please send it to me either via substack or on twitter.