Polls, errors and democracy in the 21st century

11/11/2016 by pablo.cerdeira@gmail.com

I have been working with data analysis for public policy design, public administration, and electoral campaigns for a while. And I have been following electoral polls from all around the globe during the last years.

Nowadays, the question in everyone’s minds is: what is happening with the polls? Why are they so wrong? The problem is neither caused by polls methodology nor by politicians, but by the governments. I’ll explain why.

Poll errors

We saw polls getting it wrong about the Brexit. On the election day, “YouGov predicted the Remain campaign would win, and the final three polls before the referendum all said the same thing: the UK would vote – albeit by a small margin – to stay in the EU. With hours to of, the bookies were 9 to 1 on that the UK would stay in the EU.”

The same occurred in the 2015 UK elections: 100% of the polls were wrong. In a table compiled by BBC, “92 polls showed results ranging from 17 dead heats to three polls where the gap was 6 percent between Conservative and Labour. Most polls showed Labour leading, and not one of the 92 polls predicted the 7 percent lead the Conservatives actually would achieve.”

We witnessed polls going crazy about the public referendum regarding the agreement between the Colombian Government and the Revolutionary Armed Forces of Colombia (FARC). Here in Brazil, not a single poll was able to predict that João Doria could win the São Paulo election in the first round. The examples are endless.

And then, there was Trump’s election. Nate Silver, considered one of the most influential people in the World in 2009 by Time, became known after successfully forecasting the 2008 and 2012 elections in the USA. Silver’s work is based on an analysis of 372 different pollsters results to call the outcomes. His model has worked very well, being able to successfully forecast the results in 49 of the 50 states in the 2008 U.S. Presidential election.

But for his model to work, the polls need to be reliable. And Trump’s election proved they are no longer well-grounded. In some states, Trump’s overperformance of was as great as +8.5 (Utah), or 6.6 (Ohio).

Using data collected from the polls, Silver predicted that there was a 71.4% chance of winning for Hillary Clinton, and only 28.6% forecast for Trump.

The reasons, according to the pollsters

According to some pollsters, “Women who voted for Trump might have been especially reluctant to tell pollsters”, said David Paleologos of Suffolk University.

To Geoff Garin, “many surveys had under-sampled non-college-educated whites, a group that Trump appealed to.” CNN has considered that the polls were too focused on the major cities, and people from rural areas may have been underrepresented.

Dan Healy, Managing Director of FTI Consulting, speaking on the 2015 UK election, believes there is a “Shy Tory” effect: Conservatives are less likely to reveal their loyalties than Labour voters. He also says: “Polls can be fooled by a late swing; that is, by voters who change their mind after being polled.”

Considering all those criticisms, we could appoint three main reasons for the poll failures:

1. Some groups may be reluctant to open up their opinions to the pollsters;

  1. The population samples are not well selected by the pollsters; and

  2. Instances of late swing are now more common than before.

Is this true?

Causes 1 and 2 above might explain some cases, but we should ask: if this is true now, why did we not see the problems that we are now facing reflected on polls produced five or ten years ago? Have people become more reluctant only recently? Are we now faced with a different distribution of our citizens, unusual enough to affect the samples? Further, these causes do not explain poll failures in smaller regions; take the Brexit or the São Paulo mayor election as examples.

We cannot explain the polls’ problems by simply taking a few possible discrepancies or errors in terms of reluctance or sample selection.

So, we must analyze the third reason: the late swing. Do you believe citizens are likely to change their votes at the last minute more so recently than a few years ago? If so, why?


Perhaps the occurrence of late swing is related to protest by the voters. “Early numbers suggest that voter turnout in 2016 wasn’t quite as high as everyone expected it to be”, according to Vox. A preliminary estimate holds a voter turnout of 56%, not so different from the previous elections (2004: 55.7%, 2008: 57.1%, 2012: 54.9%).

In Brazil, where voting is mandatory, the summation of blank and null votes in 2016 was the highest ever recorded.

In the UK referendum regarding the Brexit, the youth turnout “was almost double the level that has been widely reported since polling day, according to evidence compiled at the London School of Economics.” About 64% of the young voters did vote, in fact.

So, the voters are still voting. But they are more likely to use their votes as a form of protest. In Brazil, we are using blank and null votes. In the UK and the USA, the voters may be swinging their votes in the last minute to show its nonconformism.

The citizens are not worried about the result of the election. They are not analyzing the effects of a vast number of people swinging their votes to protest. To them, the vote is when their protest can be heard.

But to protest against what? And why?

People are using their votes to be heard. According to some psychologists, attention seeking is a response caused by neglect.

Voters may be using their votes as a way to call for attention. They want to be heard by the politicians; they want to take part in governmental decisions. But they are not allowed to get involved, to support or to emit their opinions in the government management. The only moment they are able to play a role is during the elections.

So, we should start thinking of new ways to get the citizens engaged in the governmental decision-making process. Not only during the elections, but also, and more importantly, after them.

Civic participation in governments

Here in Rio de Janeiro, we are attempting to do this through technology. At PENSA, we collect data from social networks such as Twitter and Waze, or from sensors such as bus GPS systems and cell phone location data, particularly with the purpose of increasing the participation of citizens in public decisions.

These databases can be very useful to bring hidden yearnings from the citizens to the attention of public administrators. Where is the heaviest traffic jam, according to our citizens? Where should we apply more of our efforts to reduce car accidents? How can we improve our mass transportation methods to answer our people’s needs better, based on the most traveled routes gathered from the cell phone location data?

When we send real-time information to our citizens through apps like Twitter and Waze, we are also trying to build a bridge between the government and society; we are seeking to bring the government closer to the people.

When we open our data to the public, as we did in the data.rio website, we are inviting the citizens to analyze data about our administration, and to help us to take better decisions.

The recent polls’ errors carry a much deeper meaning than a simple methodology mistake. They should be seen as a call from the citizens; it is a sign that we need to allow the society to take part in the government. The democratic model the citizens want cannot be reduced to an election every four or so years.

One way to accomplish this is through new technologies. There are thousands of different projects supporting this idea. However, most of them are still considered experimental or tests for future implementation. All major governments should start to look at these projects more closely if they want to reduce the public rejection they are facing all around the globe.

Pablo Cerdeira is the Head of the Center of Technology and Society - CTS/FGV and the former Rio de Janeiro's Chief Data Office