By Craig Kafura and Karen Whisler
As the results of the United Kingdom’s snap election filtered in last Friday, most headlines echoed shock: Theresa May and her Conservative Party had lost the large majority in Parliament that seemed almost guaranteed just a few weeks ago. The UK faces a ‘hung’ Parliament – the Tories hold 318 seats against the Labour Party’s 262, down from the 330 they had pre-election and below the 326 required for the party to confidently control the UK legislature.
The Tories now must attempt to form a support and consent agreement with the far-right Democratic Unionist Party—a deeply controversial decision criticized by former Conservative PM John Major—just to hold onto their political power. They’ve emerged from an election of their own choosing in a much weaker position—and Brexit talks are set to begin later this month.
What drove this shocking shift? While May’s campaign may be partly to blame—it’s largely being cast as ‘disastrous’— voters were also not satisfied with how her administration was handling key issues. In a survey conducted in May of this year, Ipsos MORI found that the two most important issues facing Britain were the National Health Service (NHS) and Brexit, according to 61 percent and 45 percent of voters, respectively. Tellingly, these are also two areas which the majority of UK voters believe the current government “dealt with badly”, regardless of political affiliation, according to a YouGov poll in May.
Perhaps just as surprising as the disastrous showing for the Tories is the failure of many political polling outlets to accurately forecast the result. Of the major polls conducted before June 8, most claimed a handy victory was in store for the Conservatives, in line with May’s original expectations. Yet this is not the first time that a majority of pre-election polls in the UK have failed to predict a vote.
In the run up to the 2015 General Election, most polls were predicting a dead heat with a hung Parliament the predicted outcome. However, the election results were surprising: the Conservatives won a decisive majority of 330 seats in Parliament. One reason for the discrepancy was lower voter turnout among the young and working-class demographics the Labour Party relies upon. As a result, many UK polls recalibrated their prediction models, weighting them to discount Labour and emphasize Conservative voter intentions. The result: raw numbers for many polls this year indicated a narrow Conservative victory, but headline predictions foretold a much larger lead.
It now seems that pollsters overcorrected for past errors by failing to take Labour voting intention seriously. Turnout to the polls on Friday surged to a 20-year high: 68.7 percent of those registered voted, many of them the very demographic that cost Labour the election two years ago. Young people were much more likely to take this election seriously, and May did herself few favors with them by refusing to engage with other party leaders in debates. A survey by YouGov in April found that 70 percent of 18-to-24 year-olds would have preferred that she taken part, even as she believed she had ‘nothing to gain’ from them.
The results chronicle her mistakes. On election day, the youth went to the polls, and they turned heavily against PM May. From 2015 to 2017, turnout for 18 to 24-year-olds rose from 43 to 66 percent, and nearly two-thirds of them voted for the Labour Party. Today age has become a far more important predictor of party alignment than even the traditional class divide of British politics.
While most pollsters missed the mark, there is one clear outlier: the YouGov modeling approach. More than a national poll, YouGov used what is known as Multilevel Regression and Post-stratification (MRP) which takes a week’s worth of national polling data, adds that data into a model incorporating demographic characteristics of past voting populations, and then predicts the vote in each of the Parliamentary constituencies. It was the only approach predicting a hung parliament, and that outlier prediction meant it incurred a lot of scorn in the weeks leading up to the election. But critics are now eating crow (or their own books) while YouGov’s pollsters and modelers celebrate a successful prediction that only 28 percent of voters are pleased with. If you’d like to know more about the technique used by YouGov, an overview of MRP can be found here. It’s written by Columbia professor Andrew Gelman, who helped develop YouGov’s model for the election.
Far from producing May’s desired “strong and stable government”, the election leaves the UK politically unsettled. Roughly half of Britons think May should stand down as Prime Minister, and Labour leader Jeremy Corbyn is telling his MPs to remain “in permanent campaign mode.” If there is a Tory/DUP deal, it will only give the Conservative government a narrow majority with which to run a very complicated negotiation over Britain’s exit from the EU.
In this environment, it’s possible that YouGov and other pollsters will get another shot at predicting a UK general election soon enough.