A well-thought-out questionnaire is the backbone of any great piece of research. However, this is often overlooked as people tend to focus on the output rather than the process of how to get there. Refining and perfecting your questionnaire with the output in mind will make the process much easier when it comes to analysing the data and will also ensure that this data is robust.
A good questionnaire can help you build a story before you even have any data, ensure high data quality, and maximise completion rates.
But how can you do this?
Top six tips for questionnaire design
Tip 1 – Know the headlines you are looking for
Having a set of desired headlines that you’d like to get from the survey as a base makes writing the questionnaire much easier. Do you have a hypothesis to validate? Do you have an impact or maturity model to create? If so, make sure you create questions that lend themselves to that specific headline or model.
The questions should support the story you’re looking to tell or the headlines you need for your campaign, and it will be easier to get the desired angles to explore in the data. The type of question used here will also be crucial in dictating the kind of data that the research generates.
Tip 2 – Understand which question type is appropriate
Use ‘select one’ when you want to know exactly which is the ‘most important’ (or equivalent), or when it’s appropriate (e.g. on an agreement scale). However, bear in mind that if you ask respondents to select one from a long answer list, it can not only be tricky to answer but can result in very split results with no clear ‘winner’.
In these situations, using ‘select all that apply’ or ‘select up to three’ is the best option. This allows you to get higher percentages whilst also providing a rank order of answers (i.e. you can still see which comes out as first, second, and third from the list). You can see a list of good and bad questions below after ‘Key Takeaways’.
Tip 3 – Be mindful of the respondent’s experience
To make it as easy as possible, try to avoid ranking questions as they are more complex than they seem for respondents to answer and don’t tend to provide high-quality, interesting data. Instead, we recommend using Likert scales (from strongly agree to strongly disagree) or asking the respondents to select their top 3 challenges. This provides better quality responses and can still be reported as the most popular or most preferred when doing PR research, for example.
Try not to ask more than 25 questions (roughly a 10-minute survey), as this can mean people lose interest and engagement, which impacts the quality of responses as the survey progresses.
Tip 4 – Cover all the bases in question options to ensure data quality
Use closed questions that have a list of ready-made responses (i.e. code frames) that respondents can tick. Understanding and researching issues related to the topic/market before code-frame creation is key to ensure a comprehensive list. If you’re in doubt about knowing all the potential ready-made options, use an ‘other’ category, particularly if you want to understand the impact of a problem or list of information sources.
We also strongly recommend including a neutral and ‘don’t know’ options as this allows for a larger breadth of responses rather than forcing participants to select options that might not reflect the truth, resulting in skewed data. Forcing respondents to choose an option they do not necessarily align with can also lead to higher dropout rates.
Tip 5 – Pick the correct format and ensure variety throughout
There are many ways that you can ensure the most engaging and varied set up for the respondent: mix it up with short and long questions; include images or video where relevant. Put some simple to answer questions among a set that require more thought.
Also, make sure to keep the formatting of the questions clear and interesting. Don’t just ask lots of grid questions (e.g., rating questions), as respondents easily get bored with these.
Tip 6 – Watch the flow of your questionnaire and know your audience
Asking questions in the right order can be the difference between good and bad data. Ensuring questions which are related to each other are asked consecutively and placing questions into sections means that respondents will be ‘in the zone’ for that type of question before moving on to the next topic.
Try to be sensitive in the way that the questions are posed and the topics that are handled. Ask yourself how you would feel if you were in the respondent’s shoes and asked that question. Adding instructional text or pages with definitions can also aid the respondent’s experience.
Key Takeaways
There are numerous benefits to having a good questionnaire but there are two key outcomes you should focus on:
- Setting yourself up to succeed – The goal of any research project is to have a great output that meets your objectives. It is pivotal to frame your questionnaire with headlines in mind and in a way that meets these objectives. The types of questions used and the context of the questions asked are very important and can vary depending on what it is you want to find out.
- Improving data quality – Here at Sapio Research, our purpose is to provide data confidence for all. A major part of this is about optimising data quality, and a good questionnaire can significantly influence this by increasing respondent engagement, ensuring the respondents views are covered, and reducing dropout rates. Quality data means quality results and should always be at the forefront of any research project.
Example Questions
Yes/No questions:
We recommend splitting the Yes / No options to provide more insight.
Don’t do this ❌
Q1. Do you currently know how to write a good questionnaire? Select one
- Yes
- No
Select all questions:
- Ensure there are enough options for the respondent to choose from
- Check whether a ‘select up to three’ question would be more appropriate to avoid respondents selecting all the options, which can result in flat data
- Give respondents an option if none of the options apply to them
Don’t do this ❌
Q2. Which of the following are the key elements of a good questionnaire? Select all that apply.
- Interesting topic
- Different types of questions
- Short length
Do this instead ✅
Q2. Which of the following are the key elements of a good questionnaire? Select up to three.
- Interesting topic
- Different types of questions
- Short length
- Good flow
- Plenty of options to choose from for each question
- Opportunities to write in own answers
- Understandable language
- Other (Please specify)
- I don’t know the key elements of a good questionnaire (Exclusive)
Select one questions:
- Ensure the options do not overlap
- Make sure all options are covered
- Give respondents another option if none of the options apply to them
Don’t do this ❌
Q3. What is your current level of expertise on questionnaire writing? Select one
- Our organisation is very good at it
- We make our best effort to write good questionnaires
- Questionnaire writing is a time-consuming task
- Questionnaire writing is fun
Do this instead ✅
Q3. What is your current level of expertise on questionnaire writing? Select one
- Expert – Our questionnaire writing is the best
- Competent – We make the best effort to write good questionnaires but can still improve
- Novice – We have had some training but can definitely improve further
- Not applicable – Questionnaire writing is something we are not trained in at all
Numerical questions:
- Ensure the bands do not overlap
- Make sure all options are covered
- Where possible, try and keep the bands consistent (one-year gap, for example)
Don’t do this ❌
Q4. How long have you been writing questionnaires? Select one
- 1 year
- 1-2 years
- 3-6 years
- 6-10 years
Do this instead ✅
Q4. How long have you been writing questionnaires? Select one
- I have never written a questionnaire
- Less than a year
- 1-2 years
- 3-4 years
- 5-6 years
- More than 6 years
Scale / Grid questions:
- Make sure the questions are not leading
- Try to keep the options neutral to avoid bias
- Try to be more specific and avoid using subjective terminology
- Allow for answering neither agree nor disagree or don’t know to avoid false data and high dropout rates
Don’t do this ❌
Q5. To what extent do you agree with the following statements? Select one per row
- My team is well-trained in questionnaire writing
- My team finds it difficult to pick up new skills
- Strongly agree
- Agree
- Disagree
- Strongly disagree
Do this instead ✅
Q5. To what extent do you agree with the following statements? Select one per row
- My business likes to support staff with personal development
- My team prefers to stick with the tasks that employees are experienced in
- Strongly agree
- Somewhat agree
- Neither agree nor disagree
- Somewhat disagree
- Strongly disagree
- Don’t know
How can we help?
Below are the trio of services that we provide when it comes to questionnaire design:
Full design: If your team is low on time and/or if you would love our expertise, we can build a questionnaire from scratch using your objectives, ideal headlines, business problems or stories for a campaign, and feedback rounds. This is also very useful when the questionnaire builds on a previous stage, such as the desk research or the qualitative stage.
Partial design: We’d recommend this for people who have some good ideas of what questions they would like to ask but need a bit of support structing them and their ideas. We will feed in with suggesting questionnaire edits that will support your objectives and carefully review at every stage to make sure that your needs are being supported with our expertise.
Questionnaire advice: This is the best approach if you have a lot of experience creating questionnaires in-house and are confident in writing questions, with little or no support needed. We’ll flag any potential changes that could improve the questionnaire that your team can choose to implement (or not), and support on developing the screening questions to define the survey audience.
