Picking the right question types
Open questions
Open questions allow you to collect qualitative insights. Respondents can write their answers in their own words and draw out whatever details they want. Open questions have no pre-defined answers or options and allow for more than a one-word answer. For example, ‘Tell us about your experience of being on a clinical trial,’ or ‘What could have been changed to improve your experience?’ are examples of open questions.
Advantages
- Allows for in depth answers
- Gather a wide range of responses
- Answers are in the respondents’ own words
- Useful when you’re exploring new area, people’s attitudes, opinions or motivations
Disadvantages
- Respondents may not give relevant answers
- May take more time for patients to complete the survey
- More time consuming and complicated for you to analyse data
Closed questions
Closed questions allow you to collect quantitative insights. Closed questions let respondents choose from predefined options or provide one-word answers. For example, ‘Is Paris the capital of France?’ or ‘What is more important to you? a) football, b) theatre, c) reading, d) other, e) prefer not to answer’ are examples of closed questions.
It’s important to keep option lists short (no more than 8 answer options). If the list is too long, respondents may not read it all, which will lead to inaccurate data. If needed, think about splitting the question into more than 1 question. You should randomise the order of the option list across participants where possible to minimise response bias.
Advantages
- Quick for respondents to answer
- Easy to compare results
Disadvantages
- You need to provide all the answers you’d anticipate from respondents
- Restricts the range of responses
- Restricts what type of information you can gather
TIP: It can be useful to follow up some closed ended questions with an open ended question to ask the respondent to explain why they chose what they did. For example:
Question 1: Do you enjoy taking surveys?
- Yes
- No
- Don’t know
Question 2: Please explain your answer:
Scale and ranking questions
Scale and ranking questions can be used to ask respondents how they feel about each item, to rate items on a scale, or to rank items in order of importance or preference. This is where answer options are redefined but on a scale. Answer options can be in words or in numbers, if the meaning of the numbers is explained.
For example, ‘How useful was the patient information provided to you at time of diagnosis: a) very useful, b) useful, c) neutral, d) unuseful, e) very unuseful, f) prefer not to say’ is an example of a scale question.
‘Please rank each of the following items in order of importance, with 1 being most important and 4 being least important: easy to read, use of diagrams and images, accessible online, available in print’ is an example of a ranking question.
Advantages
- Gives more information on each item
- Allow respondents to give more detailed answers rather than ‘Yes/No’ answers
- Provides average and measure of spread
- Scale questions allow you to ask more questions with the same measure, making it easier for respondents to answer
Disadvantages
- Takes more time to answer
- Places equal importance on each option and may force differences that don’t exist
- Can be complicated to analyse data
- Ranking questions don’t capture distance (i.e. how much more important one choice is over the other)
Things to watch out for with scale questions
- Make sure the answer scales correspond to the question
- The answer options must be balanced so that it doesn’t lead respondents to answer in a certain way
- Don’t use mid points that aren’t meaningful (e.g. ‘neither likely or unlikely’ or ‘neither important or unimportant’). 'Neutral' can be an example of a meaninful midpoint depending on the question. You can also include no midpoint.
- Make sure that the answer options don’t overlap
- Each option must be of equal magnitude
- Clearly label number scales with what the number means