Design Testing

Paul Boag

Headscape run design testing sessions. We find it a helpful way of getting sign off on a design. I know not everybody agrees with this approach so I thought I would share what we do and why we do it.

Getting sign off on the look and feel for a site can be an amazingly painful process. Everybody has an opinion on design and it seems to matter very little that they are paying you (the designer) for your expert advice. It continually amazes me that clients would prefer to design it themselves by micro managing the process, than allow us to do the job for which they pay.

The root of the problem is that design is a subjective thing. Although there are underlying principles of good design a lot of it comes down to what people personally like or dislike. This makes deciding on a design a potential stumbling block.

Now, there are numerous approaches you could take to get a design approved but the approach we have adopted at Headscape is design testing. I am not going to claim for a minute this makes the sign off process easy but it does at least smooth the way and make it slightly more objective.

What is design testing

Probably the easiest way to describe design testing is to say what it is not. It is not a scientific process like say, polling. It is not a matter of simply selecting the design that scored the best. Its is a completely subjective process where observation and intuition are more important than statistical results.

You cannot simply rely on what people say initially. It is necessary to engage with them and endeavor to understand the underlying motivation. Probably the most common question I use in design testing is “why?”. By continually asking why, you dig deeper, encouraging people to express in more detail their response to a design. By talking to them face to face you pick up something of their body language and emotional response rather than a simple statistical result.

Design testing is also not a focus group. It is vital that you engage with users on a one to one basis. Group discussions lead to less confident members being influenced in their thoughts and this undermines the results. Interacting with a website is almost always a solitary experience and so the design should be assessed in the same way.

So what exactly happens in a design test session? Well in most of the sessions we run the user is asked to participate in two exercises:

Flash Testing

The idea of a flash test is simply to judge the weighting and emphasis of the design. Are the right elements highlighted? Will the user be distracted by something of minor importance? Will they notice a key piece of content?

This is achieved by showing the user the design for a few seconds and then removing it from sight. You then ask the user to recall as many items as they can and note both, which items were spot as well as the order in which they are recalled. Screen elements that had the most impact on the user are always mentioned first.

Emotional response

The second exercise is much more subjective and takes practice to get right. This is when you get into the realms of personal opinion.

We all know it is no good asking somebody what they think of a design. The reaction you inevitably get is “I don’t like the green” or something equally subjective. However, there is value I believe in asking the user what emotional response a design generates in them.

We simply ask people to describe the site using non design related words. So they could describe the site as “funky” or “moody” but not as “too dark” or “too busy”.

Some users find this a hard challenge so it may help to give them choices just to get them started. Typically we have a list of extremes we ask them to choose between. For example is the design “conservative” or “progressive”. Is it “cold” or “friendly”. Although to some extent you are leading them at this point it is still useful data and it gets them thinking in the right way so that they can continue with their own wording.

Is design testing worth while?

I have talked about design testing before with other designers and in many cases I get a negative reaction. The perception is that the client is paying the designer for their expertise and so it should be the designer that develops the look and feel in isolation. They also express a fear that the design will be compromised somehow if the “masses” get their say.

Both of these objections are in fact valid. The designer should be making the design decisions and too many cooks definitely can spoil the broth when it comes to design. However, in the real world clients do interfere in the design process and design testing can be an effective way of minimizing the influence of a single poorly informed person.

I believe that design testing offers 4 powerful benefits.

Beyond personal preference

We all know when we don’t like a design. A lot of us even have strongly held opinions about what we like. However, none of that matters very much when it comes to a website. What matters is what the user likes. I have designed sites before which I personally hate but which I have known the target audience love because we tested it on them.

Design testing can help you focus on the user rather than your personal likes or dislikes.

Avoids design / client confrontation

I have watched too many projects grind to a halt because the client and designer cannot agree on the design. The designer feels the client should bow to his experience and training while the client is convinced that, as he is paying for the site, he should get what he wants.

These kind of stalemates can be incredibly damaging, but design testing is an excellent way of diffusing these differences.

Informing the design process

Don’t get me wrong. I am totally convinced that the designer should be in control of the design and that design by committee is the nail in the coffin of many websites. However, design testing is about informing the design process not replacing it.

In my opinion the more informed the designer is the better. By meeting real users and having the opportunity to ask them questions about the look and feel of a site, the designer can only be better informed when developing his concepts.

Minimizes design by committee

Although you could perceive design testing as design by committee, I actually think it is an effective way of preventing that. Design by committee normally happens when the client starts showing the design around to his colleagues because he doesn’t want to make the decision himself. You as the designer are rarely there and so don’t get to hear the feedback first hand or control the way the design is presented and what questions are asked.

Design testing is a control environment and yet still gives the client the confidence his or she needs. After all if the design is liked by real users, how can you argue with that.

Not perfect

So there you go, that is design testing in a nutshell. I am not claiming it is the perfect system, but we have found it very useful for keeping a project moving and getting that final sign off on look and feel.