The anonymity of online polls makes it easier for people to give their real opinions – while convenience makes them better at reaching social liberals
The EU referendum is the most important vote Britain has faced for 40 years: the citizens of the United Kingdom get to choose directly whether their future is inside or outside the European Union. Analysts are agreed that the outcome will be greatly affected by whether turnout is high or low, as the eurosceptics are the most passionate on this topic and therefore the most likely to vote. Because turnout can be influenced by perceptions of how tight the race is, polling over the next three months will be particularly important. YouGov has correctly predicted the outcome of elections not only in the UK but across the globe over the past fifteen years, until last May we - along with all other pollsters - got it wrong. It is therefore essential to know, can we start trusting the polls again?
Over the past few months online polls have shown the race neck-and-neck, while phone polls have suggested a comfortable margin in favour of 'remain'. Data analysis, including the report issued yesterday by Matt Singh, has tended to agree on the chief causes of the difference, but not on whether offline or online is nearer to the truth. Singh correctly predicted the 2015 vote, so he should be listened to with care, but we think he is clearly wrong in his latest judgments.
Singh's chief argument in favour of phone polls is that they are closer to the results of what he deems to be the 'gold standard' of polling, the British Election Study polls, which are conducted in the home, face to face. He takes as his key measure the recorded levels of social liberalism, as measured by attitudes to race and gender equality. Phone polls show greater social liberalism than online polls, with the BES somewhere in between. This greater social liberalism he takes as the decisive reason to trust phone polls more. But it is well understood that people are less likely to admit to racist and sexist attitudes when asked directly by another human being. This is called 'social satisficing', saying what one believes the interviewer wants to hear, rather than what one actually believes. Respondents are more willing to admit to unsocial attitudes online, where they feel anonymous. It is striking and particularly relevant to this EU referendum campaign that the 'gold standard' face to face methodologies under-reported the UKIP vote - the BES by 2% and the giant NatCen study by 4%. So if phone polls show even lower levels of race and gender bias than the BES, that suggests they are less accurate not more accurate than online polls, especially as the 'remain' vote is strongly correlated with social liberalism.
Singh's paper also argues that online samples underrepresent social liberals because they are harder to find. This is based on their experience that people who the BES could not reach until they had knocked on the same doors on several different occasions are more liberal. It then makes the assumption that being hard to reach at home is a proxy for being unlikely to join an internet panel. But there is no logic to that assumption. Indeed we would take the opposite view: busy socially active people find it more convenient to take part in online polls and are therefore easier to reach than by other methods.
In addition to Singh's study, the final report of the British Polling Council is published today, confirming conclusions we ourselves came to last November about what led to the failure of all pollsters in the last general election. Our errors were of sample, over-representing the votes of the more politically engaged among the young, and under-representing those of the over-70s. We have boosted the relevant demographic groups in our new sample frame. By additional recruitment and weighting, we have addressed the problems that Matt Singh assumed in his paper were still active in our polls, but which are not.
We therefore believe that YouGov is now correctly showing the true state of voting intention. But whichever methodology is closest, there is still a huge problem for all attempts to predict of the final outcome: psychological pressures will play a much larger role in this outcome than for most elections. The choice between a Labour government and a Conservative government may seem stark to many, but in reality the differences between them are rarely as clear-cut and can usually be reversed quite quickly in another election. In the case of the EU referendum, however, the decision to leave represents a massive irreversible change with unknowable effects, whether for good or ill. The deep human drive to avoid risk is likely to become increasingly powerful as we get closer to a real decision which cannot easily be changed once it is made. This strongly suggests that whatever the state of the polls today, it is impossible right now to predict the level of change yet to come. A year ago YouGov showed the Scottish people were willing to flirt with breaking up the union, but changed their minds at the last moment. YouGov correctly predicted that last minute change. We will do our best to get this one right as well.
