2015 and 2016 were not good years for polling companies in the UK, with the general failure of the industry to predict the outcome of the 2015 general election, and then a mixed bag of results in advance of the EU Referendum. The industry’s response to both those events has continued a long trend in the history of polling: innovating to address past mistakes. The inquiry into the 2015 polling miss informed the industry that it needed to improve its sampling in terms of political support and engagement, and age. By contrast, the outcome of the EU Referendum somewhat endorsed the accuracy of online polls at the expense of phone polls, with the latter struggling to reach less educated people, who were more likely to vote Leave. The good news for the industry is that the 2017 general election polls indicate that polling companies have addressed some of the problems that beset them in the preceding two years.

The underlying data gathered for the 2017 polls tended to be closer to the outcome of the general election than the adjusted headline figures that were often released, indicating three things. First, the industry has gone a long way in addressing the sampling problems identified by the inquiry into the 2015 polling miss. Second, the general shift from phone to online polling since the EU Referendum has the capacity to deliver data that is as high, or higher, quality. Third, it remains remarkably difficult to model turnout. Joining these points together, the error that afflicted some companies’ results stemmed not as much from the underlying data as from the tweaks that were applied in the weighting of data, especially to try and account for the varying likelihoods of people turning out to vote. Indeed, there was a tendency for more severe turnout adjustments to lead to worse published results. A good example here is Survation, who applied minimal adjustment to their data on the basis of likely turnout and produced the most accurate conventional polling result.

Beyond the improvements made to the quotas used by companies to ensure the representativeness of their samples, and the continued difficulty of modelling turnout, the 2017 polling results indicate an interesting new direction for the industry. They demonstrate the potential for complex models to predict the outcomes of elections with a remarkable degree of accuracy. Specifically, an approach called multi-level modelling with post-stratification (or MRP for short), trailed by Ben Lauderdale (of the London School of Economics) and Doug Rivers (of Stanford University), predicted the outcomes of 93% of seats accurately. Crucially, MRP required accurate data to work, which in this case was provided by YouGov, again indicating the important steps that companies have made in improving their sampling.

Whilst MRP demonstrates the potential rewards of innovation, both it and traditional methods also show the continued importance of treating polls as snapshots of public opinion. The change in support for the Conservatives and Labour over the course of the campaign shows how important it is not to treat the numbers on any given day as definitive. This speaks of the predictive limitations of polling; on one level it is not that impressive, having directly asked tens of thousands of people how they’re going to vote, to predict the outcome of an election only one day, or a few days, before it happens. There is hardly any causal distance between the questions asked and the outcomes predicted. It would be rather more impressive if predictions were made a year, or even a month, in advance based only on demographics, psychological dispositions, or voter preferences in other areas. This, however, remains beyond the capacity of the social sciences perhaps because, as Gary Younge has pointed out, politics is an ever-changing beast. This suggests that whilst the polling industry has continued its noble tradition of adapting in order to address its past mistakes, this tradition will have to extend into future as society continues to evolve.