Many organisations want to survey Members of Parliament. Beyond the basic requirements of writing any survey question, there are particular things to bear in mind when scripting a questionnaire to MPs. While these points can apply more broadly when it comes to survey design, they come into sharper focus when it comes to MPs, a group with a distinct perspective.
1. Make sure your questions are independent
The best questions are the ones where the MP does not know which side of the argument the person asking the question is on. Whenever you write a survey, try to take a completely independent position regardless of your organisation’s perspective on the issue involved. Equally, stop to consider whether the very premise of the question will be accepted by every MP answering the survey. This is something that can be easily overlooked and will bias your findings, so get your research agency to help you with this.
2. Be realistic in your use of jargon
While many MPs will understand the terminology you use in your questions, don’t assume that all will. There is often a misplaced belief that it is not acceptable to use technical jargon when asking questions of the public but it is ok when surveying MPs. While it may be easy to assume that MPs have greater knowledge than the public when it comes to some subjects, like the rest of us they often struggle with jargon, with acronyms particularly problematic.
3. Think about the question from the MP’s perspective
Too often we are asked to field questions designed wholly for the client’s agenda and which give almost no thought to how the MP might wish to answer. Three of the most common missteps in this area are asking them to consider an issue in a way that doesn’t allow them to answer in a realistic way, posing a question without appropriate context, or introducing unrealistic trade-offs (such as the success of one policy only being at the expense of another). The danger of doing these things is that they frame issues in an unnatural way or make unreasonable assumptions that lead to unreliable findings. In the worst cases, it means the MP doesn’t feel able to answer the question and so won’t.
4. Get the answer options right
Frequently we have had to push back on clients’ questions to MPs because the answer options may skew the results. For instance, the respondent may not be provided with all possible answers to a question or there are too many positive options and not enough negatives. Sometimes this means the more extreme potential responses are missing or unfairly combining possible responses. Whether or not the answers have been intentionally designed to frame things in a more favourable way for the client, it is essential to get your research team to play devil’s advocate with your questions and think about the breadth of possible answers.
5. Avoid using supporting evidence in questions
When asking questions to MPs, providing too much information can have a number of undesired effects. Complex definitions and descriptions, particularly at the start of a survey, may feel sensible but can be problematic. For example, if you are looking to use the data for PR purposes in order to get a headline for a story, then it can be difficult to clearly define to the media what question you asked if it includes a lengthy list of supporting evidence.
The best solution to this problem that we have found is to gather initial responses which are a little more spontaneous and then follow up with more specific questions that allow them to define the issue themselves afterwards in order to find out in more detail what they believe. This generally gives a more accurate impression of an MP’s understanding and views of a particular issue before they may become affected by the question wording.
6. Ask one question at a time
As surveys to MPs are charged ‘per question’ it is tempting to cover too many bases at once. However, to get meaningful and useful results it’s best to ask one question at a time. After all, you can’t find out how your organisation compares to others while simultaneously asking what you need to do improve its reputation. Instead, try to think about exactly the information you need and then get your research agency to help you sculpt the right question. It is better to spend a little more on research that you can use than save money by cutting corners and getting results that are unusable.
Image from PA
YouGov runs a monthly MP Omnibus, surveying a representative sample of the House every time. More details are available here.