Creating a better survey
Posted in Site content on: Friday, June 27, 2008 by Paul Boag
The web allows us to interact with our customers more than any other medium. One of the tools in our arsenal is the online survey. However, these are often badly implemented. How then can we make your surveys more effective?
Let’s be honest, surveys are one of the less personal ways of communicating with your customers online. Chat rooms, forums, and other forms of online discussion tend to create more of a dialogue, although the results are less scientific. However the survey does have its place. Let’s look at 12 says you can improve your online surveys.
Be focused
Define a clear purpose for your survey. Why are you running it? What information are you seeking to gain? How are you going to interpret the results? Avoid the temptation to throw in additional questions as this increases the likelihood of drop out.
Explain the survey
Explain to those participating why you are running the survey. Users are more likely to participate if they can see the point. If they understand your objectives it may also help them to provide answers that are more useful to you.
Show appreciation
If people can see you appreciate their opinion then they are more likely to participate. Personally I am a cynical about achieving this using incentives. I believe a personal, well written, message can be as effective. If you wish to give a gift, then do so at the end of the process as a thank you. This changes the emphasis from a payment (if you complete this survey we will give you this) to a gift (we really appreciate the time you have taken and would like to thank you.) In my opinion this reflects better on the brand. People may even encourage others to complete the survey so they too receive the thank you.
Do not distract
Consider how and when you will promote your survey. This is a fine line. On one hand you want the survey to be prominent. On the other, you need to ensure that it does not distract from users completing more important actions. Avoid popups or other intrusive methods. This only serves to alienate users. Instead consider using your mailing list or adding a small (but prominent) link to each page of your website.
Don’t make me think
Be careful not to require too much effort from your participants. The highest completion rate will come from easy to answer questions. Asking somebody about their favourite breakfast cereal is one thing, asking them what they had for breakfast last Tuesday is more challenging.
Start easy
Take a lesson from phone surveys. These traditionally start with a few simple questions that require no thought at all. Typically these are questions like age, sex and location. They have found that if they can get somebody to answer one or two questions, they are much more likely to do the whole survey. If the first question is too challenging people give up, presuming the entirely survey will require too much effort.
Use specific questions
Ensure your questions are as specific as possible. Open ended questions are harder to answer and difficult to analysis. Failing to be specific can also lead to the wrong question being answered. For example you may ask "what do you think of this site?". This could lead to comments on content and download time when you were looking for feedback on design.
Provide examples and context
When clarification of a question is required provide examples of possible answers. However, be careful that this does not bias the responses you receive. An alternative maybe to provide some context to the question. By explaining why you are asking the question the user will better understand the type of answer you require.
Avoid the non committal answer
You should give the user the option to skip a question or specify an alternative to the options provided. However, avoid allowing non committal answers. The most obvious example of this is the 1-5 rating system. The majority of people will select 3 because it is a middle of the road option and requires the least thought. Effectively 3 is a decision not the answer the question. As Zeldman says "Maybe is one option too many".
Avoid personal questions
Always keep your survey’s anonymous. An online survey is not the place for collecting names and contact information. Although it is acceptable to ask for demographic information, people are going to be reticent to give you their email address and phone number. Asking for this with significantly decrease the number of people completing the survey.
Watch your language
The way questions are worded can make a substantial difference in results. For example using the word ’should’ instead of ‘could’ has been known to alter results by up to 20%. Make sure the words you choose are as neutral as possible and even consider running A/B comparisons with alternative wording if the question is particularly important.
Following good form design
Finally, when creating an online survey carefully consider best practice for form design. This is something I have written about before so be sure to check out that article too.
6 Comments
Comments are for the discussion of this post. If you have other questions / comments then post them to the forum or send me an email
I like your point about showing appreciation. I have always found it hard to trust surveys where people were paid to participate.
Do you think that the survey applications like Survey Monkey are too impersonal?
Hi Paul,
First of all I would like to congratulate you on this great article. You have made some very good points.
I am the web designed for Kwik surveys, hence look at many surveys. Some have been well designed some are very poor.
I have found the surveys with the highest response rates are single page surveys with upto 25 Questions. People like to see how long the survey is. They exponentially lose patientience after 3 pages.
Like you mention it is very important to think about how you analyse the results. In a large survey 200+ short text answers is almost impossible to look through, where on a small survey 10 to 20 text answers can give you excellent feedback.
Keep up the great work,
Jeff.
@paul i love the way you present your points.. but i can imagine that they work better if you bundle them as ‘10 points for more effective questioning’.. perhaps further down the road you might also like to talk about creating better surveys (online version!)
@mat, if you like survey monkey u’ll prob like http://www.pollograph.com too
Can anyone recommend a survey tool which can be hosted under my domain and can be fully customised to fit in with my website design.
Many Thanks
Steve
Can anyone recommend a survey tool which can be hosted under my domain and can be fully customised to fit in with my website design.
Many Thanks
Steve
[url=http://www.cashsurveys.net]Surveys[/url]