The snap election means that polling innovations introduced after 2015’s general election are being put to the test sooner than expected
The performance of election polling has come in for a lot of criticism in recent years, some of it deserved, some of it not.
At the 2015 general election the polls were genuinely off, showing the parties neck and neck when in fact the Tories had a comfortable lead. The Brexit polls were more of a mixed bag than they are portrayed, with many commentators focusing on the telephone polls with a Remain lead. However, the majority of polls during the campaign actually had Leave ahead, and several companies correctly showed this on the eve of the vote. Certainly some polls were wrong, but a large number were correct.
The British Polling Council’s inquiry into what went wrong with the polls in 2015 concluded it was down to poor samples and most polling companies seem to agree. The ever increasing difficulty of getting people to participate in surveys meant those who did were increasingly unrepresentative, being too engaged, too educated, and too interested in politics. These issues were not being adequately corrected in the pre-2015 weighting schemes.
The effect of all of this was to skew things towards Labour. For example, the sort of young people who took part in polls went out and voted for Labour, but the sort of young people polling companies couldn’t reach ended up staying at home and not voting.
But while there is agreement about why the polls were wrong, the solutions that polling companies have adopted to correct the problems have varied. Some companies – including YouGov – have concentrated on improving their samples, through new quotas and weighting variables.
Other companies have instead tried to address the problem through changes to their turnout models. Instead of working off whether people claim they will vote, they instead base their estimates of whether respondents will vote on their demographic characteristics.
The early election on June 8th means all these innovations are going to be put to the test rather sooner than most people in the polling industry expected. Although every company is showing a large Conservative lead, as things stand the scale of the lead differs. The weekend before Theresa May announced the snap election there were two polls showing the Tories with 21 point leads and another with only a nine point margin.
While this may confuse voters (and even the media) a bit, this variation is probably good for the industry in the long term. Given companies are trying out lots of different innovations and experimental changes, the election will inevitably find that some are more successful than others. Those changes that work at the general election will be adopted and copied by other firms, while the ones that don’t will be quietly dropped.
From YouGov’s point of view, we’re confident that we’ve both correctly identified what went wrong in 2015 and the best way to cure it. We’ve invested a lot of money in improving our panel and changed our quotas and weights to try and correct those age old problems of political polls having too many ‘political’ people in them. We have focused of getting larger numbers of less politically-minded people on our panel and have added additional quotas and weights based on respondent’s education and level of political interest.
Like the rest of the polling community we are as ready as we’ll ever be. But, ultimately, we – and the rest of the polling industry – will only know if the changes have worked in the early hours of June 9th as the election results come in and we can measure our work against the actual behaviour of the voters.