What goes into a user testing script
Posted in Usability on: Friday, February 29, 2008 by Paul Boag
I have talked before about the benefits of user testing. I have discussed how to user test on a budget. Now, I want to look at some basics that go into every usability test script.
What should go in a usability test script? That largely depends on what you are testing. If you are testing design concepts then your testing will be limited to questions about the navigation and communication of core messages. You could carry out some flash testing but your options are fairly limited.
However, if you are testing a wireframe or version of your site then more options are available. In such cases, the testing is about completion of tasks. For example, could the user find the price of a particular product, or the contact details for a key member of staff.
The choice of tasks to test should be based on key activities that your personas wish to complete. Let me give you an example. Let us say we have a persona called Jane who was considering attending a health spa. The first two pieces of information Jane wants about the spa is price and availability. It is therefore logical that any testing for the spa should include tasks to find this information.
Although what is tested will vary, there is some information that should always be included. Below are highlights from a fictional transcript demonstrating what should always be covered.
Hi Jane. My name is Marcus and I am going to be running the session. Joining me is Paul. I have asked him along to take some notes as we talk. I hope that is okay.
By introducing yourself and others in the room you help to put the user at their ease. Offering coffee can help too! Be sure to explain any recording equipment in the room as this can be intimidating.
The idea of this meeting is to see if we can improve a website that is currently under development. You are going to help us test the site. Its important to understand that we are testing the site and not you. So you can relax!
By explaining to the user that you are testing the site and not them, they will behave more naturally.
I should also explain that there are no right or wrong answers so don&#’;t worry about messing up.
We need you to be honest. If you are struggling with something or don&#’;t like the way it works, say so. You aren&#’;t going to offend anybody.
If the user perceives the session as a test (with right and wrong answers) they will tell you what they think is right, rather than what they feel.
Also, many users are worried about offending the facilitator with negative comments. This is why it is important to stress that you want honest answers and no offense will be taken.
The most important thing to remember is that we need you to explain what you are thinking. Try to think out loud and talk about the various options you are considering. Before you click on any link explain what other options you considered and why you picked the one you did.
Getting the user to articulate their thoughts is fundamental to the success of the session. It cannot be stressed enough. Even though you have explained this up front, you will still need to prompt them throughout the session.
Finally, if you have any questions please feel free to ask. I might not be able to answer them straight away because this could prejudice the testing. However, I will answer them at the end.
It is important to explain why you may not answer their questions during the session. If they do ask questions be sure to address them at the end.
Let&#’;s start off with something easy. Can you tell me a bit about yourself? Tell me about your job?
It is always good to begin a session with some simple personal questions such as family status, age and job title. This helps build the users confidence and provides useful background information.
Tell me a bit about your computer experience. How confident do you feel using a PC? Do you use them for work? What about at home?
How much do you use the internet? What kind of sites do you use regularly and find most useful?
Building up an understanding of the users computer and web experience provides context for the session. It also indicates how representative they are of the target audience.
Okay, lets talk about the site. Its a site for a health spa. Before I show you the site I want to ask about your expectations. What do you think a health spa website should look like and what information would it contain?
It is helpful before revealing the site to ask users about their expectations. If the expectations do not meet the reality it can cause confusion. Asking about expectations also provides opportunity to find out more about what users want from the site.
The session would then continue to address issues specific to the stage of development the website was currently at, This part of the test is primarily either &#”;do they understand what they are seeing&#”; questions or task completion.
4 Comments
Comments are for the discussion of this post. If you have other questions / comments then post them to the forum or send me an email
This is a good outline; I wish I had this when I was doing my first usability testing. That said, there’s two things I’d suggest:
First, even though you made it explicitly clear that the site is being tested, not the user, I used the word “test” in my copy and, despite the lengthy explanation I provided to the user that we were testing the site and its effectiveness (like you included in this article), the user still ended up asking how they were doing and if they were passing. The next time after that, I removed “test” from all of my copy except for instances of “usability test” and I didn’t have that problem again.
Second, I’d suggest telling the user in the middle and at the end of the test that the feedback you’re getting is really useful and you appreciate their time (or something like that). I’ve noticed that users feel more comfortable after telling them that they’re doing exactly what you need them to do.
I’ve found the two things above useful, but others mileage may vary. Thanks again for the site and the podcast. :)
Thanks a lot for the article and the Podcast about test Scripts. Greetings, Werbeagentur
Steve Krug FTW
This is really helpful to me. I’m new to the world of user testing, but have been intrigued with some results that I’ve had. I’m wondering: is it standard practice to do user testing on the current site, in the redesign process? I’m doing that with a site I’m currently in the initial phase of redesigning, and I’m finding the information to be very helpful in narrowing in expectations and actual use rather than my client’s desires for the site.
Also, the people that I’m using for the test are current “consumers” of the product, and so they have a current perception of the site and an understanding of its structure. Is that bad? How can I make it work? Any suggestions?
As always, I love the podcast and appreciate your time.