Research Q&A's

Q. Many millions of people do not have internet access. How can you possibly reflect the views of the public as a whole?

A. When the internet was new, reliable surveys were indeed impossible to conduct, simply because too few people had access to it. But now it has spread to every significant demographic group, with 62% of GB Adults logging on. Online researchers are able to reach sufficient numbers of women as well as men, over-60s as well as under 30s, people on below-average as well as above-average incomes. National surveys are therefore conducted to represent the public as a whole.

Q. Yet surely it is easier for telephone polling companies to achieve a representative sample, because almost everyone has a telephone these days?

A. Not necessarily. There is a variety of reasons why telephone polling companies have difficulty obtaining a truly representative sample. They tend to reach people who spend the most time at home and who are most willing to talk to strangers. They are less likely to reach people who are out a lot, or often use voicemail to screen incoming calls, or don't like talking to strangers, or mainly or entirely use a mobile phone. (Telephone research companies use only landline numbers to reach people.) Telephone polling companies generally achieve only 15 interviews for every 100 residential numbers they dial.

Q. But perhaps the people in your samples, all of whom are online, may differ from those who are not online in significant respects, even if they resemble the general public demographically?

A. All companies face the challenge of taking the views of the people they can reach and estimating the views of the people they can’t. Even if they get the demographics right, how can they be sure that the people who do complete surveys are like the rest of the public? It is precisely to respond to this challenge that YouGov seeks to produce results that are not just demographically representative but also attitudinally representative. As well as weighting our raw data to ensure that our figures conform to the profile of the nation by age, gender, social class and region, we also weight our data by newspaper readership, and often by past vote. For example, we ensure that our published figures contain 22% who are Sun or Star readers and 4% % who are Guardian or Independent readers.

Q. You mention "weighting". Newspaper reports of surveys often say that the data have been “weighted”. What does this mean?

A. Almost all surveys involve weighting, whether they are conducted online, face-to-face or by telephone. This is to ensure that the published results properly reflect the population they seek to measure. For example, men comprise 48% of the electorate and women 52%. The raw figures in a well-conducted survey will be close to this, but not necessarily match these numbers exactly. Suppose the raw figures contain 50% men and 50% women. YouGov's computer would slightly "downweight" the replies given by the men (so that the replies of 50 men count as if they were 48) and slightly “upweight” the replies given by women (so that the replies of 50 women count as if they were 52).

In practice, the task is more complex than this, as matters such as age, social class, region and newspaper readership, as well as gender, have to be considered simultaneously. This is a task for YouGov's computer, which adjusts the raw data to take account of all these factors.

Q. Aren’t your surveys self-selecting – and therefore likely to be skewed to people who have strong views about the subject of any given poll, rather than representative of the public as a whole?

A. No. When we seek the views of the general public, we select which respondents we want to survey. Only those selected are able to complete the questionnaire. When we email them we do not give the subject of the survey. Furthermore, our incentive system is designed to attract people who are not interested in the subject in question, as well as those who are passionate about it. This means that we experience little or no "drop-off" when a multi-topic survey shifts from one subject to another.

Q. How big are your samples – and how do these compare with those of polls using traditional methods?

A. Most of YouGov's polls of the general public achieve samples of at least 2,000. Among other companies conducting regular polls for the media, MORI’s monthly political tracking polls also have samples of around 2,000, while ICM and Populus normally poll 1,000 people. The headline voting intention figures - which eliminate respondents who say "don’t know" or "won't vote" or, in the case of MORI, ICM and Populus, are judged unlikely to vote – are based on samples, typically, of 1,500 for YouGov, 1,000 for MORI and around 550 for ICM and Populus.

Q. Why does sample size matter?

A. Because the risk of random sampling error is related to sample size: the smaller the sample, the greater the risk of such error. On a sample of 550, we can be sure that, 19 times out of 20, the true figure – that is, the figure that would have been obtained had the whole population been polled using the same methods – is within 4% of the published figure. Random error on a sample of 1,000 is up to 3%, on 1,500 up to 2.5% and on 2,000 up to 2%. Larger samples also allow the views of subgroups, such as women voters or Conservative supporters, to be measured more accurately.

Q. Isn’t there a danger of a "panel effect": that is, by being polled regularly, YouGov's respondents become conditioned by the process and cease to be representative of the electorate as a whole?

A. It is rare for any YouGov respondent to be asked the same or similar question (e.g. voting intention) very often. Any panel effect is therefore likely to be negligible. However, we monitor this from time to time by comparing the results from "fresh" with "repeat" respondents. So far we have found no variation.

Simple logic tells us that, if it were true that repeatedly asking the same question of the same individual could for example change the way someone was going to vote, it would no doubt be a technique widely used by political parties. This is not the case.

Q. How can you tell that your respondents don’t simply give quick, random, slapdash answers, rather than their genuine views?

A. As with any polling company, we cannot completely guarantee that not a single respondent will lie or play silly games. However, there is no evidence that this is a real problem. From time to time we ask some respondents classification questions they have answered before to check for consistency – and find little or no evidence of anyone trying to take YouGov for a ride. There is evidence that many people are more honest when answering questions anonymously via a computer than talking to a stranger. They are also under no time pressure when completing surveys online. They can take as long as they want – which is one reason why online surveys are better than telephone or face-to-face surveys for asking complex questions that need time for thought.

Q. What is to prevent people – perhaps extremists of one kind or another – from signing on to YouGov's pool in order to influence YouGov's results?

A. When we conduct political and public policy surveys, the great majority of the people we survey are those we have proactively recruited via other carefully-selected websites. We monitor closely the minority who register with YouGov by visiting our site. If there is any sudden deviation in the pattern of such recruits, we retain the option of excluding them from our surveys.

It should be borne in mind that any organisation attempting to “move” our figures by, say, ten percentage points would have to infiltrate more than 5,000 people. Any attempt to do this would quickly be detected.

Q. Your surveys sometimes contradict those produced by companies using traditional methods. Doesn't this prove that your methods are unreliable?

A. No. All it proves is that different methods may produce different responses. In itself, this does not prove which (if either) is valid. In fact, such differences occur relatively infrequently. Normally – for example, attitudes to the Iraq war and public responses to the Hutton inquiry – when YouGov and other companies have asked similar questions at around the same time, our published figures are much the same.

However, there are two areas where significant differences do persist. The first concerns voting intention. YouGov's figures tend to report slightly more support for the Conservatives, and slightly less for Labour, than other companies. On this, the recent record at elections tends to support YouGov's methods. In 2001, for example, we predicted Labour’s lead over the Conservatives to within one percentage point. Every other company overstated Labour’s lead, some of them by significant amounts. In the elections to the Scottish Parliament in 2003, YouGov again was alone in predicting the Labour and Conservative shares to within one percentage point. Three other companies, all polling by telephone, all significantly overstated Labour's share and understated what turned out to be the Conservatives’ share.

The second subject on which YouGov differs from its rivals concerns taxation. We consistently find greater hostility to tax rises than our competitors. We believe that this is as a result of online surveys producing more honest answers. Our findings are consistent with the results of three local referendums on the council tax. None of them has produced a majority for paying enough extra tax to improve local services. Indeed, in two of the three places (Bristol and Croydon), majorities have chosen no tax increase at all, despite warnings that services would be reduced. For some years politicians, both Labour and Conservative, have argued that (conventional) polls have been wrong to detect an appetite for higher taxes. YouGov's surveys suggest that the politicians may have been right all along.

Q. Why does YouGov consistently seem to produce results more favourable to the Conservatives than other polls?

A. Given YouGov's record for accurate predictions, and the habit of almost all other polls in the recent elections understate Conservative support, perhaps the question ought to be: why do other polls consistently produce results LESS favourable to the Conservatives?

We believe that one major reason concerns a theory known among pollsters and political scientists as “spiral of silence”. This suggests that, for more than a decade, some people have been reluctant to admit to a stranger that they might vote Conservative. The numbers involved are small – probably no more than 2 or 3 per cent of any national sample; but this is enough to give a misleading picture of the state of the main parties, especially in a fairly close race (such as the 1992 general election).

If this theory is right, then it is likely that YouGov polls are more accurate because we have no “interviewer effect”, as people responding to our surveys are filling in their questionnaires on a computer screen, rather than talking to another human being.

Q. How can you show that YouGov's polls are consistently accurate?

A. The only way any polling company can demonstrate its accuracy is when its results can be compared with real events. There have been five occasions when YouGov has predicted the outcomes of such events in Britain. In each case we have come close to the outcome. As well as the 2001 general election and 2003 Scottish Parliament elections already discussed, YouGov also correctly "called" the 2005 Conservative leadership election, the 2002 Pop Idol contest, the 2002 London borough elections and the 2007 Scottish Parliament elections regional vote.

Q. Who ensures that YouGov polls are 'honest and accurate'?

A. YouGov is a member of the British Polling Council and abides by its rules. YouGov is also registered with the Information Commissioner, and the majority of YouGov’s employees are members of the Market Research Society.

The British Polling Council (www.britishpollingcouncil.org) is an association of polling organisations that publish polls. The objectives of the Council ensure standards of disclosure designed to provide consumers of survey results that enter the public domain have an adequate basis for judging the reliability and validity of the results.

YouGov is bound to respond in full to any bona fide enquiries about specific published polls.


Political - Frequently Asked Questions (FAQ)



Are YouGov part of the government?

No. Not at all. We are an independent commercial organisation, completely separate from the government and listed on the Alternative Investment Market (AIM).

 

Does the government have a say in the work that you do?

No. We are an independent commercial organisation. The government does sometimes commission us to conduct polls, in the same way as many other commercial clients. In all cases we are committed to neutrality and clients have no influence on the data that is collected or the results that are produced.

 

Does YouGov only do political research?

No. Although we are most well-known among the public for our political research, it actually only makes up less than 10% % of YouGov’s revenue in the UK. We have over 250 people working in the London office alone – only eight of them are in the political research team.

 

Is it biased to only work with one newspaper?

We have a commercial arrangement to provide polling exclusively for one national newspaper, though this work is usually then picked up by other papers. As with all our research, our polls for newspapers are not written from any political viewpoint, but are meant to be social scientific efforts to measure public opinion. Newspapers may back a political party with a particular social viewpoint but we don’t. While specific clients may in the end have their own partisan goals, the surveys we conduct are not intended to influence your opinions, but to measure them.

 

Isn’t YouGov run by Conservatives / Labour supporters?

Inevitably the political team is made up of people who are interested in politics, but they represent a wide range of political views – both to the left and the right. More widely, other people within the organisation also sometimes have political backgrounds. For example Stephan Shakespeare, the CEO, is a former Conservative candidate, while Peter Kellner, the former company President, and Marcus Roberts, YouGov’s International Projects Director, have both been significant figures in the Labour party for many years. As a commercial research company our job is to accurately, independently and objectively measure and report the views and opinions of the public at large.

 

Are the political team all white men who went to public school?

No. The members of the political team come from a range of different backgrounds. They are certainly not all male or all white and the majority went to comprehensive schools. They are the children of, for example, an academic, a builder and a bookmaker. When they were younger they wanted to grow up to be things like Olympic swimmers, film directors and train drivers – not politicians. Regardless of background or personal opinions, in all cases the members of the YouGov team are committed to accurately measuring and reporting the views and opinions of the public at large.

 

I took a survey but don't see all of the results published. Why do you only show some of the results?

Each year YouGov conducts a huge number surveys for a wide variety of clients. The vast majority of these are for private use by different groups, companies and organisations. Such surveys are not published. The decision on whether to make all or part of a survey public, and when to do it, rests entirely with the client.

 

Are the rumours I have heard about a poll result true?

We never comment on any results or surveys that have not been published, nor do we ever comment on polls that were not conducted by us. All results of our published political polls will be available on the YouGov website and can be downloaded via our archive. Any speculation on what the results might be for a poll that has not been published are nothing more than that: simply speculation.

 

Do you ever refuse to work with a particular client?

Yes, regularly – but never for ideological reasons. We work with think tanks, campaign organisations, pressure groups and political parties from all sides of the political spectrum. However, all clients have to run what YouGov regards as fair, neutral, balanced questions. We work directly with clients on this, but if they insist on running questions that do not fit this criteria then we will refuse to work with them.

 

Aren’t the polls always wrong?

No – YouGov has an excellent track record compared to actual outcomes both in the UK and the rest of the world: most recently the election of Jeremy Corbyn and the victory of Sadiq Kahn in the London Mayoral contest. We also had an average lead of 1% for Leave in our final week of polling for the European Referendum. Before that, it was our polls that were first to correctly predict the rise of Ukip, the collapse of the Liberal Democrats and the surge in support for the SNP during the last parliament. We also correctly predicted the outcome of the Scottish Independence Referendum. [Include links for each of these]

 

Aren’t your polls biased because anyone can take part?

Anyone can register to take party in YouGov surveys, but only those people we then contact within a specific sample are allowed to take part in our published polls – and they are only allowed to take part once. The questions on the Take Part section of the YouGov website allow anyone to participate, but those results are separate from our published polls. For more information about Take Part polls please click here