Another election, another round of irrationality about polling. On the Conservative Home site (where this article originally appeared) there have been any amount of attacks on YouGov, including a suggestion from one commenter that Conservative activists would soon be suing us. The last time we saw this was from Ken Livingstone’s team, who said they would make a formal complaint against us to the Market Research Society. Why? Because we showed Boris Johnson comfortably winning the London Mayoralty, while Mori had Livingstone ahead. Pundits said, “This time YouGov will be proved wrong. This is the Big Test.” Our final pre-election poll showed Boris 6% ahead. The others had a tie, or a Livingstone lead. Boris won by 6%. Surprise surprise, there was no complaint to the MRS.
Now it’s some Conservatives who are complaining, because YouGov shows a lower Conservative lead than the other pollsters. The critics attribute this to some tweaks we have made to our sampling and weighting (see here). But surely anyone should understand this: you have to continuously tweak these things in order to stay accurate. It’s because we adapt the methodology that we have remained so accurate over our ten years of polling. It’s because the industry failed to adjust when things were changing that they go things so wrong in the past. For example, most pollsters use some form of past vote recall in order to weight their data. Past vote recall has some bias in it, and that bias changes from a few months after the last election to a few years after. Does anyone suggest there should be no adaptation? YouGov has a panel methodology, which means that we can compare what the same people say and do over many years. This helps us to be better at adapting our methodology (see here for our full record, compared to other pollsters and actual results.) Time will tell if we’re right this time.
But consider the graphs below. They show our 2005 general election tracker, with comparisons to those of Mori, ICM, NOP and Populus, and the actual result. What do you notice? True, NOP pipped us with their final poll. But look at the movement! In their final polls, all of them trend sharply toward YouGov’s numbers!
Maybe the same will happen this time, maybe it won’t. Maybe we’ll turn out to be closest, maybe we’ll be furthest. Impossible to say. But people should base their pronouncements on evidence, not on fantasy. Can anyone really believe – as some have actually claimed – that either I (a Conservative supporter) or my colleague Peter Kellner (a Labour supporter) would deliberately distort our results in order to please one side or another, and thereby knowingly damage a company that has built up its business in 12 countries on the basis of its record for accuracy? Now that’s plain silly.
