Government » American Politics of Public Opinion » How Is Public Opinion Measured?

Problems In Polling

Problems in Polling

For a number of reasons, polls may not produce accurate results. Two important factors a polling company faces are timing and human nature. Unless you conduct an exit poll during an election and interviewers stand at the polling places on Election Day to ask voters how they voted, there is always the possibility the poll results will be wrong. The simplest reason is that if there is time between the poll and Election Day, a citizen might change his or her mind, lie, or choose not to vote at all. Timing is very important during elections, because surprise events can shift enough opinions to change an election result. Of course, there are many other reasons why polls, even those not time-bound by elections or events, may be inaccurate.

Created in 2003 to survey the American public on all topics, Rasmussen Reports is a new entry in the polling business. Rasmussen also conducts exit polls for each national election.

Polls begin with a list of carefully written questions. The questions need to be free of framing, meaning they should not be worded to lead respondents to a particular answer. For example, take two questions about presidential approval. Question 1 might ask, “Given the high unemployment rate, do you approve of the job President Obama is doing?” Question 2 might ask, “Do you approve of the job President Obama is doing?” Both questions want to know how respondents perceive the president’s success, but the first question sets up a frame for the respondent to believe the economy is doing poorly before answering. This is likely to make the respondent’s answer more negative. Similarly, the way we refer to an issue or concept can affect the way listeners perceive it. The phrase “estate tax” did not rally voters to protest the inheritance tax, but the phrase “death tax” sparked debate about whether taxing estates imposed a double tax on income.

Many polling companies try to avoid leading questions, which lead respondents to select a predetermined answer, because they want to know what people really think. Some polls, however, have a different goal. Their questions are written to guarantee a specific outcome, perhaps to help a candidate get press coverage or gain momentum. These are called push polls. In the 2016 presidential primary race, MoveOn tried to encourage Senator Elizabeth Warren (D-MA) to enter the race for the Democratic nomination. Its poll used leading questions for what it termed an “informed ballot,” and, to show that Warren would do better than Hillary Clinton, it included ten positive statements about Warren before asking whether the respondent would vote for Clinton or Warren.

The poll results were blasted by some in the media for being fake.

Photo A shows Joseph P. Kennedy, Elizabeth Warren, and Barney Frank. Image B shows Hillary Clinton at a podium.

Senator Elizabeth Warren (a) poses with Massachusetts representatives Joseph P. Kennedy III (left) and Barney Frank (right) at the 2012 Boston Pride Parade. Senator Hillary Clinton (b) during her 2008 presidential campaign in Concord, New Hampshire (credit a: modification of work by “ElizabethForMA”/Flickr; credit b: modification of work by Marc Nozell)

Sometimes lack of knowledge affects the results of a poll. Respondents may not know that much about the polling topic but are unwilling to say, “I don’t know.” For this reason, surveys may contain a quiz with questions that determine whether the respondent knows enough about the situation to answer survey questions accurately. A poll to discover whether citizens support changes to the Affordable Care Act or Medicaid might first ask who these programs serve and how they are funded. Polls about territory seizure by the Islamic State (or ISIS) or Russia’s aid to rebels in Ukraine may include a set of questions to determine whether the respondent reads or hears any international news. Respondents who cannot answer correctly may be excluded from the poll, or their answers may be separated from the others.

People may also feel social pressure to answer questions in accordance with the norms of their area or peers.

If they are embarrassed to admit how they would vote, they may lie to the interviewer. In the 1982 governor’s race in California, Tom Bradley was far ahead in the polls, yet on Election Day he lost. This result was nicknamed the Bradley effect, on the theory that voters who answered the poll were afraid to admit they would not vote for a black man because it would appear politically incorrect and racist.

In 2010, Proposition 19, which would have legalized and taxed marijuana in California, met with a new version of the Bradley effect. Nate Silver, a political blogger, noticed that polls on the marijuana proposition were inconsistent, sometimes showing the proposition would pass and other times showing it would fail. Silver compared the polls and the way they were administered, because some polling companies used an interviewer and some used robo-calling. He then proposed that voters speaking with a live interviewer gave the socially acceptable answer that they would vote against Proposition 19, while voters interviewed by a computer felt free to be honest.

While this theory has not been proven, it is consistent with other findings that interviewer demographics can affect respondents’ answers. African Americans, for example, may give different responses to interviewers who are white than to interviewers who are black.

Chart shows the support of marijuana legalization by the type of poll conducted. When using a live operator poll, opposition is about –2 for Reuters/lpsos, about –1 for PPIC, and about –4 for Field Poll. The results from robo-polls show favorability at about 14 for Survey USA (April), about 10 for Survey USA (July) and about 16 for PPP. At the bottom of the chart, a source is cited:

In 2010, polls about California’s Proposition 19 were inconsistent, depending on how they were administered, with voters who spoke with a live interviewer declaring they would vote against Proposition 19 and voters who were interviewed via a computer declaring support for the legislation. The measure was defeated on Election Day.

[Attributions and Licenses]


This is a lesson from the tutorial, American Politics of Public Opinion and you are encouraged to log in or register, so that you can track your progress.

Log In

Share Thoughts