The United States presidential election of 2016 is scheduled for Tuesday, November 8. In the meantime, we know that in almost every race for 36 years, the eventual nominees have won either Iowa or New Hampshire.
While Hillary Clinton won by a razor slim margin in Iowa, Bernie Sanders took home New Hampshire by a wide margin. With the Republicans, it seems increasingly likely that Trump could win the nomination.
Mr Sanders promises free education in public universities. He wants to have government rather than private insurers to pay health care bills.
This could cost US $14 trillion over a decade, and would result in new taxes, costing most workers 8.4 percent of their income.
Workers might approve of such changes. But Mr Sander’s plans would have no chance of making it past Congress, even in one with a majority of Democrats.
Nevertheless, he won 60 percent of the vote in New Hampshire.
This represented one of the biggest victories in a contested Democratic primary.
Of course, polls play an important roll during any election, including US Presidential (see chart). But can we trust these polling data?
As the above graphic shows, polls gave Governor Kasich less than 5 percent. Nonetheless, he raked in 16 percent of votes in the New Hampshire primary of the Republican party. Trump hoovered up 35 percent of the vote as predicted by pollsters (see above graphic from The Economist, January 30, 2016, p.17).
The above illustrates that polling is a tough job, especially if one intends to get it right. In 2014, pollsters struggled with these challenges during the Scottish referendum and Swedish elections. Both times they got it wrong.
The same happened during the UK elections on May 7, 2015. The final polls showed Labour and the Conservatives neck-and-neck at 34 percent.
But when the final numbers where in, David Cameron’s Tories ended up 7 percent ahead of Ed Miliband’s Labour party. As pollsters have pointed out, however, they got the numbers right for the smaller parties. Too bad, their prediction was outright wrong for those fighting for the post of running the country, i.e. becoming Prime Minister…
So if you see a statement like the one below, will you believe it?
Polls indicate, Hillary Clinton is leading Bernie Sanders by 30 points in South Carolina
We discuss this below in more detail.
1. Getting the right sample size is very expensive
In early states such as Iowa and New Hampshire pollsters conducted more polls for 2016 than in 2012, when the last presidential race was happening. But the average sample size has fallen. According to the Financial Times, for New Hampshire the averages look as follows:
2012 – average pool of Republican voters interviewed was 590
2016 – average is 490
Of course, the margin of error of plus or minus 5 percent are common with such polls. Governor Kasich was predicted to get less than 5 percent. With a margin of error of 5 percent he could have gotten 10 percent but he actually got 16 percent.
In this case, the pollsters were vastly off, illustrating that sample size matters. In other words, does polling 490 Republicans justify the conclusions drawn by the authors?
[su_custom_gallery source=”media: 2470″ limit=”7″ link=”image” target=”blank” width=”529px” height=”322px” Title=”National Statistical Service of Australia – calculating sample size – it works” alt=”National Statistical Service of Australia – calculating sample size – it works”]
National Statistical Service, Australian Bureau of Statistics – Sample size calculator
[su_box title=”Polls are getting ever more costly thus end up with smaller samples” box_color=”#86bac5″ title_color=”#ffffff” radius=”5″ width=”px 700″ ]
1. 2016 US Presidential Election
Pollsters now make 30-35 calls to complete a single interview.
Ten years ago, a pollster called ten people to get one to participate in the poll or study.
To interview 1,000 voters — only 400 of which may be likely to vote Republican — the pollster now has to dial up to 35,000 numbers.
2. UK polling
In the UK, today it takes about 15,000 phone calls to get 1,000 interviews.
About 20 years ago, 2,000 calls were needed to get 1,000 replies.
The above shows, depending upon the country, we may need to make anywhere from 10,000 to just about 40,000 calls to end up with a sample of 1,000 respondents.
But in the case of an election, we also need to make sure that those who answer are also those who will go cast their vote. This challenge is discussed below.
2. Selecting the wrong sample is a growing risk
Polls are conducted over two or three days, meaning we try to interview those who are more easily contacted. This may happen over the internet or via phone, but such work makes it hard to arrive at a sample surveyed that is representative of all voters.
[su_custom_gallery source=”media: 2469″ limit=”7″ link=”image” target=”blank” width=”528px” height=”317px” Title=”National Statistical Service of Australia – calculating sample size – Youth Barometer UBS” alt=”National Statistical Service of Australia – calculating sample size – Youth Barometer UBS”]
National Statistical Service of Australia – calculate your sample size required for the relative standard error you desire
For instance, May 2015 British election data indicate that opinion polls failed to reach the harder to find Tory voters. In turn, estimates of the vote share attributed to Labor were skewed.
According to the British Social Attitudes survey, Labour was six points ahead among respondents who answered the door on the first visit. However, looking at those that required three to six home visits to be interviewed, the Tories enjoyed an 11-point advantage. Adjusting for social class and age, first time respondents are less conservative. “Busy” respondents are more likely to be so – but harder to chase for pollsters.
Hence, for pollsters it is not easy to get those “busy” people that might vote for a particular party as outlined below. Another challenge is that more and more people no longer have a landline. They can only be reached by mobile number (see also FT mentioning Martin Boon, director of ICM research). Because of this trend, in the US polls are increasingly conducted using mobile phone numbers to call.
3. Getting likely voters is a challenge for pollsters
Selecting those that are likely to vote for your political poll (see graphic below) is important. Finally, the views of voters and nonvoters are often very different, as was the case in 2014 in the US mid-term elections.
[su_custom_gallery source=”media: 3274″ limit=”7″ link=”image” target=”blank” width=”757px” height=”394px” Title=”The Pew Research Center: distinguishing likely voters from nonvoters is critical for pollsters.” alt=”The Pew Research Center: distinguishing likely voters from nonvoters is critical for pollsters.” ]
As well, for the 2015 UK elections, under-30s generally lean left, but very often fail to turn out on polling day. The pollsters, however, reached an atypical group of youngsters, who were unusually engaged with politics and committed to voting.
Another factor that can bias polls is that they are often based on internet polls. These tend to use volunteers to sign up to online panels. They may be drawn at random to participate. Nevertheless, the underlying group of people is self-selecting. Thus, data collected using a sampling method known as random digit dialing or “RDD” results in better data sets (i.e. random-probability samples representative of the population).
As this shows, getting those to participate in a poll that are likely to vote continues to be a challenge.
Here are three challenges pollsters will continue to grapple with and we voters must keep in mind when studying poll results.
[su_box title=”3 challenges for pollsters” box_color=”#86bac5″ title_color=”#ffffff” radius=”5″ width=”px 700″ ]
1. Polls are expensive!
Ever fewer people want to participate in consumer surveys or election polls. Hence, it takes thousands of calls to get a useful sample. In turn, inferences can be made from such a sample about the election outcome.
But just the calls may take about 400 hours. In turn you get 1,000 replies and still have to pay for data analyses and a write-up. A costly exercise that is not getting cheaper.
2. Getting a representative data set is nearly impossible
As the 2015 UK election illustrated, it takes a greater effort to get Tory voters than others.
Cultural differences may also limit the applicability of Pew Research’s findings in the US whereby “…better-educated people tend to be more available and willing to do surveys than are those with less education.”
However, the current solution of using online non-probability survey panels makes polling results less accurate.
3. Identifying and polling likely voters is hard
It is tough to get accurate readings to predict election outcomes by using self-selected online panels.
YouGov knows this is a sensitive issue and does not publish any answers to research methodology questions on its blog. For instance, the voters over 70 who broke heavily for the Tories were not reflected in YouGov’s online panels used for predicting the 2015 UK elections.
Those under 30 years of age are less likely to cast their ballot. If your panel or poll includes those overeager millennials, who were unusually engaged with politics, your findings will be skewed.
Polling models will continue to be improved, but surprises will also continue to happen.
Nevertheless, polls are the best way of trying to figure out what the election result is going to be next time an election, primary or referendum comes around. So go and vote.
Join the 3,000+ organizations using the DrKPI Blog Benchmark to double reader comments in a few months while increasing social shares by 50 percent – register now!
Mr Sander’s foreign policy promise boils down to one thing: he will not start a foreign war. The rest is a replay of 1960s student radicalism. He intends to convert the US into a Scandinavian social democracy. Mr Trump would make America great again. He thinks the US has been screwed by its allies. If he needs expertise, since he has none, he will hire experts. In the meantime his solution is to “bomb the shit out of ISIS.” That is about it for his foreign policy.
Unfortunately, both Sanders and Trump are arch-fantasists with the ability to recruit voters. Both will spell extremism as far as US foreign policy is concerned. The system, dominated by the Democrats and Republicans, has always rejected the political extremes. The nation has benefited from a deep political stability, which has contributed greatly to its economic strength and global power. If America’s immunity to extremism is ending, the whole world will feel the consequences. Not a good prospect for anyone, is it?
This post: Yes Virginia, pollsters really are wrong
Join the conversation
- Do you have an example of a great poll / study?
- I refused participating in a poll / consumer survey last week. You?
- Do you think pollsters will get it right in the 2016 US elections?
- US Primaries: Clinton and Trump or Sanders and Trump? What do you think?
Of course, I will answer you in the comments. Guaranteed.