Research Q&A's

 

Q. Many millions of people do not have internet access. How can you possibly reflect the views of the public as a whole?

A. When the internet was new, reliable surveys were indeed impossible to conduct, simply because too few people had access to it. But now it has spread to every significant demographic group, with 87% of British adults having access to internet in the home (in comparison, only 81% of people now have a landline telephone). Online researchers are able to reach sufficient numbers of women as well as men, over-60s as well as under 30s, people on below-average as well as above-average incomes. National surveys are therefore conducted to represent the public as a whole.

 

Q. You mention "weighting". Newspaper reports of surveys often say that the data have been “weighted”. What does this mean?

A. Almost all surveys involve weighting, whether they are conducted online, face-to-face or by telephone. This is to ensure that the published results properly reflect the population they seek to measure. For example, men comprise 49% of the adult population and women 51%. The raw figures in a well-conducted survey will be close to this, but not necessarily match these numbers exactly. Suppose the raw figures contain 50% men and 50% women. YouGov's would slightly "downweight" the replies given by the men (so that the replies of 50 men count as if they were 49) and slightly “upweight” the replies given by women (so that the replies of 50 women count as if they were 51).

In practice, the task is more complex than this, as matters such as age, social class, region and education, as well as gender, have to be considered simultaneously to make sure the sample is representative across all the targeted variables.

 

Q. Aren’t your surveys self-selecting – and therefore likely to be skewed to people who have strong views about the subject of any given poll, rather than representative of the public as a whole?

A. No. When we seek the views of the general public, we select which respondents we want to survey. Only those selected are able to complete the questionnaire. When we email them we do not give the subject of the survey. Furthermore, our incentive system is designed to attract people who are not interested in the subject in question, as well as those who are passionate about it. This means that we experience little or no "drop-off" when a multi-topic survey shifts from one subject to another.

 

Q. How big are your samples?

A. Most of YouGov's polls of the general public use samples of 2000 or 1500 respondents.

 

Q. Why does sample size matter?

A. Because the risk of random sampling error is related to sample size: the smaller the sample, the greater the risk of such error. On a sample of 550, we can be sure that, 19 times out of 20, the true figure – that is, the figure that would have been obtained had the whole population been polled using the same methods – is within 4% of the published figure. Random error on a sample of 1,000 is up to 3%, on 1,500 up to 2.5% and on 2,000 up to 2%. Larger samples also allow the views of subgroups, such as women voters or Conservative supporters, to be measured more accurately.

 

Q. Isn’t there a danger of a "panel effect": that is, by being polled regularly, YouGov's respondents become conditioned by the process and cease to be representative of the electorate as a whole?

A. It is rare for any YouGov respondent to be asked the same or similar question (e.g. voting intention) very often. Any panel effect is therefore likely to be negligible. However, we monitor this from time to time by comparing the results from "fresh" with "repeat" respondents. So far we have found no variation.

Simple logic tells us that, if it were true that repeatedly asking the same question of the same individual could for example change the way someone was going to vote, it would no doubt be a technique widely used by political parties. This is not the case.

 

Q. How can you tell that your respondents don’t simply give quick, random, slapdash answers, rather than their genuine views?

A. As with any polling company, we cannot completely guarantee that not a single respondent will lie or play silly games. However, there is no evidence that this is a real problem. From time to time we ask some respondents classification questions they have answered before to check for consistency – and find little or no evidence of anyone trying to take YouGov for a ride. There is evidence that many people are more honest when answering questions anonymously via a computer than talking to a stranger. They are also under no time pressure when completing surveys online. They can take as long as they want – which is one reason why online surveys are better than telephone or face-to-face surveys for asking complex questions that need time for thought.

 

Q. What is to prevent people – perhaps extremists of one kind or another – from signing on to YouGov's pool in order to influence YouGov's results?

A. When we conduct political and public policy surveys, the great majority of the people we survey are those we have proactively recruited via other carefully-selected websites. We monitor closely the minority who register with YouGov by visiting our site. If there is any sudden deviation in the pattern of such recruits, we retain the option of excluding them from our surveys.

It should be borne in mind that because of the size of our panel any organisation attempting to “move” our figures by a noticeable amount would have to infiltrate tens of thousands of people. Any attempt to do this would quickly be detected.

 

Q. Who ensures that YouGov polls are 'honest and accurate'?

A. YouGov is a member of the British Polling Council and abides by its rules. YouGov is also registered with the Information Commissioner, and is a corporate member of ESOMAR.

The British Polling Council (www.britishpollingcouncil.org) is an association of polling organisations that publish polls. The objectives of the Council ensure standards of disclosure designed to provide consumers of survey results that enter the public domain have an adequate basis for judging the reliability and validity of the results.

YouGov is bound to respond in full to any bona fide enquiries about specific published polls.

 

Q: Are YouGov part of the government?

A: No. Not at all. We are an independent commercial organisation, completely separate from the government and listed on the Alternative Investment Market (AIM).

 

Q: Does the government have a say in the work that you do?

A: No. We are an independent commercial organisation. The government does sometimes commission us to conduct polls, in the same way as many other commercial clients. In all cases we are committed to neutrality and clients have no influence on the data that is collected or the results that are produced.

 

Q: Does YouGov only do political research?

A: No. Although we are most well-known among the public for our political research, it actually only makes up less than 10% of YouGov’s revenue in the UK. We have over 250 people working in the London office alone – only eight of them are in the political research team.

  

Q: Isn’t YouGov run by Conservatives / Labour supporters?

A: Inevitably the political team is made up of people who are interested in politics, but they represent a wide range of political views – both to the left and the right. More widely, other people within the organisation also sometimes have political backgrounds. For example Stephan Shakespeare, the CEO, is a former Conservative candidate, while Peter Kellner, the former company President, and Marcus Roberts, YouGov’s International Projects Director, have both been significant figures in the Labour party for many years. As a commercial research company our job is to accurately, independently and objectively measure and report the views and opinions of the public at large.

 

Q: I took a survey but don't see all of the results published. Why do you only show some of the results?

A: Each year YouGov conducts a huge number surveys for a wide variety of clients. The vast majority of these are for private use by different groups, companies and organisations. Such surveys are not published. The decision on whether to make all or part of a survey public, and when to do it, rests entirely with the client.

 

Q: Are the rumours I have heard about a poll result true?

A: We never comment on any results or surveys that have not been published, nor do we ever comment on polls that were not conducted by us. All results of our published political polls will be available on the YouGov website and can be downloaded via our archive. Any speculation on what the results might be for a poll that has not been published are nothing more than that: simply speculation.

 

Q: Do you ever refuse to work with a particular client?

A: Yes, regularly – but never for ideological reasons. We work with think tanks, campaign organisations, pressure groups and political parties from all sides of the political spectrum. However, all clients have to run what YouGov regards as fair, neutral, balanced questions. We work directly with clients on this, but if they insist on running questions that do not fit this criteria then we will refuse to work with them.