A dispute between two environmental scientists is creating a controversy over how much methane is leaking from natural gas production and is contributing to global warming.

In a new report, Touché Howard, a methane gas expert and air quality consultant, says the flaws he found in a commonly used methane detector caused an acclaimed 2013 study to underestimate the amount of methane emitted by natural gas production. Howard’s paper was published today in the peer-reviewed journal Energy Science & Engineering.

The 2013 study was considered a landmark in methane research because it was one of the first times that oil and gas companies allowed independent scientists to take direct measurements at well sites and other equipment on company property. Led by David Allen, a chemical engineering professor at the University of Texas-Austin, the study’s researchers sampled 150 natural gas production sites around the U.S and extrapolated their results to a national leak rate for the industry.

Much of the Allen team data, derived by monitoring methane leaks from pumps, valves and other equipment, came from the Bacharach Hi-Flow Sampler, a portable instrument that measures methane emissions.

In the study published today, Howard said the Bacharach device can fail to correctly identify the amount of natural gas leaking into the air. When the flaw occurs, it always results in an underestimate, he said, and it’s nearly impossible to detect the problem while it’s occurring. “It wasn’t their fault” that this happened, Howard said of Allen’s research team.

If correct, the new report raises questions about the validity of countless other measurements taken by the same instrument since 2003. Natural gas companies often use the Bacharach to report their methane emissions to the Environmental Protection Agency. The calculations feed into a national greenhouse gas inventory.

“If Howard’s right, we’ll need to review other emission estimates used in EPA inventories,” said Robert Jackson, an earth science professor at Stanford University who studies methane leaks. “We need to sort this out as quickly as possible.”

In an email, EPA spokeswoman Enesta Jones said the agency will assess Howard’s paper along with other studies “as a part of our routine review of new information and data for potential incorporation in the GHG Inventory.”

Allen’s research was the first paper published from an ambitious series of methane studies organized by the Environmental Defense Fund, a non-profit that supports efforts to thwart global warming. EDF launched the $18-million project in 2011, with joint funding from foundations and the natural gas industry.

EDF started the project to fill a research gap. At the time, there was little data on the amount of methane coming from the natural gas industry, and getting exact numbers is crucial to understanding whether the natural gas boom will accelerate or stall global warming.

Although gas-fired power plants only release half as much carbon dioxide (CO₂) as coal plants, the extraction and production of natural gas releases unknown amounts of methane. Over a 20-year time period, methane is 86 times more heat trapping than CO₂. Methane’s potency is reduced to 34 times that of CO₂ over 100 years, because methane is naturally removed from the atmosphere more quickly than CO₂.

The EDF project joins other studies funded by universities and regulators. But EDF’s work is unique in scope, with 16 studies that will cover every step of natural gas extraction, production, transportation and storage. (Jackson has worked on several EDF methane studies, but none that involved Allen or Howard.)

Allen’s study was the first of many papers to come out of the EDF project. The subsequent studies have produced a mixed bag of results, with some showing higher methane emissions than expected, and others lower. One of the most recent studies found 50 percent more methane from Texas’ Barnett Shale than estimated by the EPA.

Howard was involved in the Barnett study, as well as several others run by EDF. But he said his critique of Allen’s work has created tension—a statement confirmed by emails provided by Howard showing correspondence between himself, EDF staff and Allen’s research team.

Howard is in a unique position to challenge Allen’s results. Howard began his consulting career in 1988, working for industry and government clients. In the 1990s, he developed a gas-sampling technology that allowed people to take speedier measurements of leak rates. Bacharach, Inc. later obtained the rights to Howard’s technology and used it to create the Bacharach Hi-Flow Sampler. It was first sold commercially in 2003.

A Bacharach representative declined to provide sales numbers. Thomas Ferrara, a colleague of Howard’s, estimates there are about 500 Bacharachs used in the U.S. today, and each costs around $20,000.

Howard wasn’t involved in the development of the Bacharach sampler, and he had never used one until early 2013, when he teamed up with Ferrara to investigate the instrument’s accuracy. Ferrara is group manager of air quality services at GHD, an engineering consulting firm. He and Howard have known each other since the 1980s, when they earned their Master’s degrees in civil and environmental engineering from Washington State University.

What began as a passing interest in the sampler would lead Howard to write his latest paper. The study went through two rounds of comments from three anonymous peer reviewers before it was accepted for publication. Howard said their feedback substantially improved his work. Still, Howard expects pushback.

“Nobody is going to be crazy about this,” he said. “So the burden of proof is on me to make it as clear as a bell.”

Howard believes he’s stirred up additional controversy by challenging the work of a prominent academic. According to his online biography, Allen has written six books and more than 200 research papers. He recently finished a two-year stint as chair of EPA’s Scientific Advisory Board.

After examining Howard’s study, Jackson, the Stanford professor, said it’s “too early to say whether some of the Allen values are wrong. Howard’s analysis suggests that they may be, however.”

If Allen’s team has “systematically underestimated methane emissions, we need to know right away,” Jackson said. Several other scientists approached by InsideClimate News declined to comment on Howard’s paper due to the sensitive topic.

Critical Flaw Is Found

Howard and Ferrara became interested in the Bacharach in October 2012, while at a scientific conference in Boulder, Colo. Ferrara said they met an EPA scientist there named Eben Thoma, who described some difficulties he’d had with the Bacharach. Thoma confirmed Ferrara’s description of their meeting through an EPA spokeswoman.

Thoma documented his problems with the sampler in a conference paper he co-authored in 2012. According to the paper, Thoma and his colleagues found the Bacharach consistently underestimated methane emissions.

Around the same time, a gas industry scientist involved with the EDF studies also raised questions about the Bacharach, Howard said. Both events inspired Howard and Ferrara to rent several samplers and test them in the field. By Fall 2013, they concluded the flaw was caused by a problem with the sensors, and most often occurred when the Bacharach sampled natural gas with a low methane content.

Although methane is the main component of natural gas, the composition of the gas varies based on geology. Some of the “raw” natural gas that comes out of gas wells may only have 60 percent methane, combined with ethane and other hydrocarbons.

The Bacharach uses two sensors to analyze methane leaks: The first one measures natural gas concentrations in the air between 0 and 5 percent, which are caused by smaller leaks, while the second sensor measures concentrations above 5 percent that are caused by larger leaks.

Under certain conditions—especially with low-methane natural gas—the second sensor failed to kick in, so the instrument only registered a small leak even if the actual leak rate was much higher. At one point, two separate Bacharach samplers tested by Howard and Ferrara recorded natural gas concentrations in the air of 1 to 6 percent, when the actual concentrations were between 7 and 73 percent.

When Allen’s paper was published in Proceedings of the National Academy of Sciences in September of that year, Howard noticed something odd: Most of the high emissions came from raw natural gas with very high methane content. In fact, there were about 7 to 9 times as many large leaks at sites where the natural gas was more than 91 percent methane than at sites where the methane content was less than 91 percent.

It didn’t make any sense, because the chemical composition of natural gas has a very small impact on how quickly it leaks out of valves, wells and other equipment, Howard said. And the same pattern existed for different types of well pad equipment, across all geographic locations sampled in the study.

That’s when Howard realized the pattern could be explained by a failed Bacharach sensor. He approached Allen, hoping to investigate the problem together and perhaps issue a joint statement or correction about the 2013 study.

But Howard said he was “stonewalled” by Allen’s research team. In July 2014, for instance, Howard and Ferrara completed a nondisclosure agreement (NDA) that would have allowed them to view unpublished raw data from Allen’s team. The two consultants said the University of Texas had asked for the NDA as a prerequisite for viewing the data, which would have provided greater insight into Allen’s 2013 paper—but UT didn’t sign its half of the agreement, so Howard never saw the data.

Howard provided InsideClimate News with a copy of the incomplete NDA. When InsideClimate News showed the document to Allen, he said he “disagree[s] with the suggestion that the University of Texas was unwilling to enter into an NDA.”

When asked to elaborate, a UT media representative said InsideClimate could file an open records request.

Allen said his team has “tried to be as responsive as we possibly can be” to Howard’s concerns. He disputes the idea that his study was affected by a problem with the Bacharach. The dataset was bound to show a wide range of emission rates given regional differences in regulations and other factors, he said. “You would not expect leak rates to be the same,” he said in an interview.

Howard said he never expected identical leak rates, but the persistent lack of high emissions for low-methane natural gas makes no sense without a failed Bacharach sensor. He also considered other theories. For instance, Colorado has strict methane regulations for the gas industry, which could impact how companies repair equipment leaks. But when Howard disregarded the Allen data gathered from the Rocky Mountain region, he found that high-methane sites were still four times as likely to have larger leaks than low-methane sites.

By August 2014, Howard had determined Allen wasn’t going to cooperate on a joint study. So Howard wrote up his concerns about the Bacharach in a paper for the Journal of the Air & Waste Management Association. Published this spring, the paper was co-authored with Ferrara and Amy Townsend-Small, a geology professor at the University of Cincinnati. Their study identified the sampler’s problem and a possible solution: The underestimates vanished when the Bacharachs were calibrated frequently and installed with updated software.

Since their conclusions came from testing just five Bacharachs—including one used by Allen’s team for the 2013 study, which malfunctioned until it was re-calibrated—the authors said more research is needed to fully understand the problem.

Tom Ryerson, a methane researcher at the National Oceanic and Atmospheric Administration, said Howard’s research may inspire scientists to re-examine their data for possible Bacharach impacts.

“Having two nice peer-reviewed papers in the literature should have people sit up and take notice,” said Ryerson, who isn’t involved in any of the EDF studies. “Maybe this is a good cautionary tale for everyone.”

Steven Hamburg, chief scientist at EDF, said he couldn’t comment on the specifics of the Bacharach sampler. Scientific integrity is one of EDF’s “core values,” he said, and Howard’s paper is “a classic case of allowing the scientific process to work through.”

“However this process ultimately shakes out, the final difference remains one of degree – whether the emissions are too high, or way too high,” Hamburg said in an email. “And nobody is disputing that they can and should be reduced. To us, that’s the most important outcome from this body of work.”

Jim Rutherford, Vice President of Products at Heath Consultants, which distributes the Bacharach, said he doesn’t believe the instrument has a software problem. Rutherford said the instrument was originally intended for use with processed natural gas that’s usually more than 90 percent methane—not raw natural gas with lower methane content.

As the instrument is increasingly used at upstream natural gas sites, he said, it raises questions about the limitations of the instrument and its modified use.

Doug Keeports, president of Bacharach Inc., said his company will update the user manual to recommend daily calibrations instead of the current guideline of calibrating at least once a month.

Experienced users know that more frequent calibrations are usually a good idea, Rutherford said, “so it’s just clarifying it in the manual.” Rutherford said his company has approached industry partners and the EPA to discuss possible further testing.