Creating a better survey

Paul Boag

The web allows us to interact with our customers more than any other medium. One of the tools in our arsenal is the online survey. However, these are often badly implemented. How then can we make your surveys more effective?

Let’s be honest, surveys are one of the less personal ways of communicating with your customers online. Chat rooms, forums, and other forms of online discussion tend to create more of a dialogue, although the results are less scientific. However the survey does have its place. Let’s look at 12 says you can improve your online surveys.

Be focused

Define a clear purpose for your survey. Why are you running it? What information are you seeking to gain? How are you going to interpret the results? Avoid the temptation to throw in additional questions as this increases the likelihood of drop out.

Explain the survey

Explain to those participating why you are running the survey. Users are more likely to participate if they can see the point. If they understand your objectives it may also help them to provide answers that are more useful to you.

Show appreciation

If people can see you appreciate their opinion then they are more likely to participate. Personally I am a cynical about achieving this using incentives. I believe a personal, well written, message can be as effective. If you wish to give a gift, then do so at the end of the process as a thank you. This changes the emphasis from a payment (if you complete this survey we will give you this) to a gift (we really appreciate the time you have taken and would like to thank you.) In my opinion this reflects better on the brand. People may even encourage others to complete the survey so they too receive the thank you.

Do not distract

Consider how and when you will promote your survey. This is a fine line. On one hand you want the survey to be prominent. On the other, you need to ensure that it does not distract from users completing more important actions. Avoid popups or other intrusive methods. This only serves to alienate users. Instead consider using your mailing list or adding a small (but prominent) link to each page of your website.

Don’t make me think

Be careful not to require too much effort from your participants. The highest completion rate will come from easy to answer questions. Asking somebody about their favourite breakfast cereal is one thing, asking them what they had for breakfast last Tuesday is more challenging.

Start easy

Take a lesson from phone surveys. These traditionally start with a few simple questions that require no thought at all. Typically these are questions like age, sex and location. They have found that if they can get somebody to answer one or two questions, they are much more likely to do the whole survey. If the first question is too challenging people give up, presuming the entirely survey will require too much effort.

Use specific questions

Ensure your questions are as specific as possible. Open ended questions are harder to answer and difficult to analysis. Failing to be specific can also lead to the wrong question being answered. For example you may ask "what do you think of this site?". This could lead to comments on content and download time when you were looking for feedback on design.

Provide examples and context

When clarification of a question is required provide examples of possible answers. However, be careful that this does not bias the responses you receive. An alternative maybe to provide some context to the question. By explaining why you are asking the question the user will better understand the type of answer you require.

Avoid the non committal answer

You should give the user the option to skip a question or specify an alternative to the options provided. However, avoid allowing non committal answers. The most obvious example of this is the 1-5 rating system. The majority of people will select 3 because it is a middle of the road option and requires the least thought. Effectively 3 is a decision not the answer the question. As Zeldman says "Maybe is one option too many".

Avoid personal questions

Always keep your survey’s anonymous. An online survey is not the place for collecting names and contact information. Although it is acceptable to ask for demographic information, people are going to be reticent to give you their email address and phone number. Asking for this with significantly decrease the number of people completing the survey.

Watch your language

The way questions are worded can make a substantial difference in results. For example using the word ‘should’ instead of ‘could’ has been known to alter results by up to 20%. Make sure the words you choose are as neutral as possible and even consider running A/B comparisons with alternative wording if the question is particularly important.

Following good form design

Finally, when creating an online survey carefully consider best practice for form design. This is something I have written about before so be sure to check out that article too.