How to Pilot a Questionnaire

Last Updated: September 15, 2021 Approved

If you're conducting a large survey or questionnaire for a research study or paper, you need to pretest your questionnaire on a smaller scale to make sure it's going to work for your purposes. To pretest, or pilot, a questionnaire, choose a representative sample of your larger survey group and have them take the questionnaire. Then you can make improvements where necessary based on their feedback.

Part 1

Part 1 of 3:

Selecting the Sample

1
Identify your target group. Ideally, the sample you use to pilot your questionnaire should be members of the same group that you intend to target for the full study. Define your target group in terms of age, gender, geographic location, and any other factors relevant to your study.
- Identify factors that will likely impact people's ability to understand the instructions and the questions themselves.
- For example, even if a respondent's educational level has no relevance to your overall study, it will impact how they are able to read and comprehend the written language.
2
Choose 5 to 10 people to test. Your sample doesn't have to be tremendously large for you to get a good sense of whether the questionnaire and your methods of administering the questionnaire are going to be effective.^{[1]
X
Research source}
- The size of your sample may depend more on the diversity of your target group than its size.
- You may want a larger sample if your target group includes, for example, immigrants from several different countries.
Advertisement
3
Look for similarly situated people. In some cases, it won't be possible to pull a sample group directly from the actual target group for your study. Look at the description of your target group and search for similar people.^{[2]
X
Research source}
- For example, suppose you have a questionnaire for factory workers in a specific geographic area, but you don't have the time or resources to go there for a pilot.
- You could have factory workers of similar age and background who happen to work at a closer factory take your pilot test.
4
Cover the range of your target group. Particularly if you're conducting a large survey, you want to make sure each subgroup of your overall target group has a chance to be part of the pilot. You want to make sure different subgroups still understand the questions the same way.
- For example, if you're testing 15-25 year olds, you need at least one tester who is 15 or 16 years old, and at least one who is 24 or 25.
- If your target group includes people who are not native English speakers, include at least one or two non-native English speaking people in your pilot test.
- If people in your target group come from diverse educational backgrounds, make sure your sample includes people who have less education as well as people with advanced degrees.
5
Do as much as you can. You may not have the time or resources to do a larger pilot of your questionnaire before you start using it for your study. When time or resources are limited, you should still try to go through the questionnaire with at least one person.^{[3]
X
Research source}
- When you do the initial planning for your project, include a pilot in your scheduling and your budget.
- Even if you have limited resources to commit, completing a pilot could save your project a lot of money by uncovering errors that would cause the ultimate questionnaire to fail.

Part 2

Part 2 of 3:

Completing the Questionnaire

1
Train your data collectors. If you're going to pilot a questionnaire, it's important that the questionnaire is administered to the testers the same way you intend to administer it to the target group.
- If you have people helping you, make sure they all distribute and collect the questionnaire using the same methods you will use in the real study.
- This ensures the results of the pilot aren't affected by some different manner of delivery or collection.
- You may want to give your data collectors a field test before you complete the pilot to make sure everyone is fully trained and doesn't deviate from their instructions.
2
Invite testers one at a time. When you pilot a questionnaire, you won't have very many testers in your sample. If they take the questionnaire separately you have the opportunity to focus on each individual.^{[4]
X
Research source}
- Individual testing also allows you to interview each sample group member immediately after they complete your questionnaire.
- This gives them a chance to talk about the experience while it's still relatively fresh in their minds.
3
Observe the testers answering the questions. While you'll be asking your pilot testers directly about their experience completing your questionnaire, you'll get a lot of uncensored information from studying their body language.^{[5]
X
Research source}
- You can have someone in the room to observe directly, or you can monitor on closed-circuit television.
- If someone will be in the room, be aware of the effect their presence may have on the people responding to the questionnaire.
- If no one will be in the room with your actual subjects when they answer the questionnaire, you shouldn't have anybody in the room with your pilot testers.
- Set up a closed-circuit camera to record testers as they answer the questionnaire.
4
Ask testers to think out loud. One of the things you're looking for when you pilot a questionnaire is whether your respondents will be able to follow the instructions you've provided. If testers think out loud, you can see where instructions are confusing.^{[6]
X
Research source}
- It's especially helpful to have pilot testers say what they're thinking if you're observing over closed-circuit television, because they might forget details that would be important to you.
5
Solicit additional feedback. As soon as possible after each sample tester completes your questionnaire, sit down and have an interview with them to find out more about their experience responding to your questionnaire.
- Ask them specifically about any questions or instructions that confused them or gave them difficulty.
- If you were observing through a closed-circuit video, ask your testers about body language you noticed while they were responding to your questionnaire.

Part 3

Part 3 of 3:

Implementing the Results

1
Enter the pilot questionnaire answers. Once all of your pilot testers have completed your questionnaire, enter the data in a computer program or on a spreadsheet exactly as you plan to do for the full study.^{[7]
X
Research source}
- This process also allows you to find any possible issues with the methods you plan to use to interpret and analyze your data.
- Using the same methods of analysis and interpretation, treat the pilot exactly the same as the full study.
2
Analyze the results. Part of the value of a pilot study is that you can see if the full project is worth doing. Based on your analysis of the data collected in the pilot study, you can often figure out if it's even worth offering the questionnaire to the full target group.^{[8]
X
Research source}
- The results of even a small group of respondents can give you a good idea whether the questionnaire will be effective at fulfilling the goals or intent of your overall study.
3
Identify basic formatting or data entry errors. Having a few people respond to your questionnaire may draw attention to errors that make a question difficult to understand or to answer. These basic errors are relatively easy to fix.^{[9]
X
Research source}
- For example, the columns for some responses may be misaligned, so that your respondents can't figure out which place to mark for the option they want. This can be easily corrected.
4
Address logistical challenges. The pilot can reveal problems with the methods you intend to use to distribute, administer, or collect the questionnaire. You have the opportunity to correct any inefficiencies that could cause significant problems once you start the full study.^{[10]
X
Research source}
- For example, use the pilot to plan how respondents should enter and leave so the people who've finished the questionnaire won't cross paths with people who haven't taken it yet.
5
Correct problematic questions. As a result of the pilot, you may become aware of questions that are confusing or even offensive to respondents. Doing the pilot gives you the opportunity to fix these questions so they'll better achieve your objectives.^{[11]
X
Research source}
- If the pilot has revealed that a question is unnecessary, redundant, or overly difficult, you may want to discard it.^{[12]
  X
  Research source}
6
Conduct a second pilot. If the pilot only revealed minor errors, you typically can fix these and move on to the full study. However, if significant changes were required after the pilot, you may want to have a second pilot for the revised questionnaire.^{[13]
X
Research source}
- If you're running low on time and resources, you could do a smaller, more informal pilot just to give the questionnaire a test run.