Since the polling error last year there have been a wide variety of different responses from the polling industry. Some of the most candid, interesting (and occasionally pessimistic) have come from Martin Boon at ICM, the company that did most of the heavy lifting of moving the industry forward from the errors of 1992.
Since 1992 ICM have traditionally been a telephone company, but more of their recent EU polling has been conducted online. As Martin says “More work has been undertaken on our online processes, given our view that the future of telephone polling is somewhat bleak, for reasons both linked to its own ability to produce representative samples in an age when the public has radically redefined its relationship with the telephone (landline in particular) and because the cost and practicality of telephone polling is problematic for vote intention work.”
His latest commentary is as interesting as ever (and well worth reading in full here - http://us11.campaign-archive2.com/?u=fbcf81e4dd2761d48aba0b6da&id=14df1d3e14). It describes some of the problems that ICM have encountered in conducted polling online, and the issues that come from fast respondents. They are familiar issues from our experiences in the early years of online polling, but thankfully ones that YouGov no longer face.
Martin explains how in ICM’s online polls they send out invitations to complete that survey, but that the quotas for different age, class and gender groups quickly fill up with those people who are very eager to respond (who in this case are more likely to be Leave supporters), leaving no space for the Remain supporters who respond more slowly.
A decade ago we used to see a similar pattern of Conservative supporters being quicker to respond than Labour supporters. In 2009, however, YouGov switched our sampling methodology completely and eliminated the issue. Rather than manually inviting respondents to specific surveys, in most cases we rely on our panel and sampling software to manage our sampling. The system automatically sends out a steady flow of emails inviting YouGov panellists to come and complete a non-specific survey. When a respondent clicks on the link (be it immediately, hours later, or perhaps days later) they come into the YouGov system and are allocated to a survey that needs someone matching their demographics. This means all our surveys will have a mix of people who respond quickly and who respond slowly, and that some of the people who take a YouGov survey will, in fact, be responding to an invitation that was sent before the survey was even written.
ICM are tackling the problem by splitting up their invitations into smaller chunks and adding an additional weight based on how quickly people respond. We wish them every success – ICM they are a company whose work we admire, but we want to make clear that it is not an issue that affects YouGov polling.
