A new industry-sponsored poll is resorting to biased push-poll tactics in an apparent attempt to offset the growing body of research that shows Americans support the U.S. Environmental Protection Agency (EPA)’s proposed carbon pollution standards for coal-fired power plants. The so-called “Partnership for a Better Energy Future,” a self-described industry association that includes the American Petroleum Institute and the National Mining Association, has released new polling they claim shows voters nationwide and in swing states are wary of the EPA’s Clean Power Plan. But a look at the actual research documents reveals a blatantly-biased survey and a lack of transparency about the survey sample.
According to the Partnership for a Better Energy Future’s press release, the survey conducted by Paragon Insights “finds that Americans have major concerns about the EPA’s proposed greenhouse gas regulations and are unwilling to pay even a dollar more for energy in exchange for these new rules.” But while the survey documents look legitimate and detailed, this research breaks two of the cardinal rules of public opinion research: being transparent about research sampling methodology and avoiding bias in question wording and sequence.
One of the first steps in survey analysis is to double-check that the demographics of the final survey sample (i.e. the group of people who completed a poll) closely matches those of the population of interest. The Partnership for a Better Energy Future, nor their pollsters at Paragon Insight, are transparent about the demographic breakdowns of their sample. The population of interest for this research was “likely voters,” for which there is no standard demographic profile to compare a survey sample against. Some informed judgement calls are necessary when sampling from this population, making it even more important to release demographic breakdowns in a survey release. The omission of this information is puzzling and could be interpreted as suspicious.
As any survey researcher will tell you, question wording and sequence are very important, as they influence how survey respondents interpret poll questions and how they answer them. And biased question wording or sequence will usually yield biased survey results. And given that this industry poll employs both biased question wording and biased question sequence, the results are automatically suspect.
To illustrate how biased the Partnership for a Better Energy Future’s poll is, consider its main finding that a plurality of voters (47 percent) oppose “the EPA regulations” while 44% support them. If you look at the parts of the survey questionnaire which were released, you will see that these results come from a question that appears after respondents hear a battery of arguments for and against the “Obama administration’s regulations to reduce carbon emissions from power plants.” Instead of following industry standard and asking for respondents’ opinions before influencing them with qualitative information, this survey first presented twice as many arguments against it than arguments for it. While the negative arguments included questionable (and scary) claims like “the new regulations could increase your yearly household energy costs by as much as $130” and “could lead to job losses in your state,” the positive arguments were far less personalized. One reads “the new regulations could persuade other countries to join the fight against global climate change or the environment” and another claims “the regulations could reduce global carbon emissions by as much as 1.5%.” Conveniently absent is any mention of the significant public health benefits of reducing harmful pollution from coal-fired coal plants. Our own research has found that protecting public health is seen as one of the most-convincing reasons to support the EPA’s Clean Power Plan.
These arguments influenced how these respondents would respond to this question. If I did not know much about about the EPA’s Clean Power Plan before taking this survey, I would be more likely to say I opposed it after hearing just three “pros” and six “cons.” Especially if the list of “pros” did not include the top reason Americans support the plan: its positive impact on the health of America’s families.
There’s good reason to believe these biases yielded flawed data, as these survey results run counter to every major publicly-released poll this year. Earlier this year, a national survey conducted by Quinnipiac University found that 58% of registered voters support federal government limits on “the release of greenhouse gases from existing power plants.” These findings are consistent with a Wall Street Journal/NBC News poll, which found that two-in-three American adults (67%) supported the EPA setting “strict carbon dioxide emission limits on existing coal-fired power plants.” Another 57% said they would approve of a proposal to reduce greenhouse gas emissions from companies, even if it would lead to higher utility bills for consumers.
--Grace McRae, Sierra Club Polling and Research Strategist