As regular readers of this blog will know, we are proud of YouGov’s record as Britain’s most accurate polling company. One reason for our record is that we regularly review our sampling and weighting methods and change them when necessary.
Our latest review, ahead of the coming general, has led to three changes.
First, we have refined our sampling techniques. We found that response rates from Labour identifiers were slightly, but consistently, lower than response rates from Conservative identifiers. This was easily corrected at the weighting stage; but, on the principle that the less weighting that needs to be done, the better, we have made slight changes to our sampling frame to reduce the need for subsequent weighting.
Second, we have always divided our Labour identifying respondents by those who were ‘loyal’ in 2005 (Labour identifiers who voted Labour) and those who were ‘disloyal’ (Labour identifiers who did NOT vote Labour in 2005). Before we launched our daily polls for the Sun and Sunday Times four weeks ago, we were conducting daily polls for internal testing purposes. We found that when results varied from day-to-day, this sometimes flowed not from external events but from differences in the mix between loyal and disloyal Labour respondents. We have therefore decided to eliminate this variation by weighting separately for loyal and disloyal Labour identifiers. The overall proportion of Labour identifiers remains unchanged.
Those first two changes have already been implemented; neither has made any systematic difference to our results. However, our third change does make a difference to our figures for Scotland. We have found that in Scotland, our party ID figures have not kept pace with the cumulative changes that have occurred since the SNP came to power in 2007. From this week, we are adjusting our sampling and weighting methods to take account of this. These will typically reduce Labour’s recorded share of support in Scotland by two-to-three percentages points and increase the SNP’s recorded share by a similar amount.
We have also recalculated recent voting intention surveys conducted in Scotland. These changes do not change the big picture: Labour has regained a substantial lead north of the border in recent months as support for the SNP has slipped; and, as it happens, these changes make little difference to any seat projection that are made applying uniform swings in Scotland.
It should also be noted that these changes to our Scottish data have no bearing on GB-wide polls (or the Scottish crossbreak in GB-wide results tables), as party-ID weights for these polls are applied on a GB-wide basis.
It should also be noted that all reputable polling companies review and adjust their methods from time to time. Traditional polling companies had to do so after their inaccurate polls in the 1992 and (with the exception of ICM) 1997 general elections. This has led to a wide variety of methods for measuring voting intention. This is because each polling company has to use its judgement. For example, most polling companies ask respondents how they voted at the previous general election; they then have to decide whether, and if so by how much, to weight their data to allow for ‘false memory syndrome’ – the tendency for some people to misremember whether, and sometimes how, they voted. This inevitably involves a degree of judgement; and the judgement may change from time to time.
We do not anticipate making any further changes to our sampling and weighting methods before the coming election. However, in keeping with YouGov’s commitment to transparency, if we find we need to, we shall say so.
