Testing design: Testing users impressions of a design

Paul Boag

We all know that testing the sites we produce is important and there are well established techniques for testing usability. But what about testing aesthetics? What about testing design?

It would be wrong to have a series on design methodology without looking at the subject of testing design. One of the fundamental principles behind Headscape’s design approach is to design with data.

Design is subjective. What I like you may well hate and vice versa. Design can often lead to arguments between client and designer about what direction the design should take.

The normal route for resolving this problem is to focus on what the user wants, rather than what the client or designer thinks is best. However, often all this does is shift the argument about what the design should be like to what the user will want! The only way of resolving this is to test.

Testing the usability of a site is relatively easy. There are tried and tested ways of doing this which I will cover later in this series. However, what about aesthetics? How do you test the best colours, typography, styling etc? How do you ensure that your design is communicating the right feel and personality to your users?

The first step is to know what you want to learn about the aesthetics.

Asking users what they think isn’t enough

It is not enough to just show users a design and ask them if they like it or not. This approach creates two problems.

First, whether they like a design is not the only criteria by which to judge it. For example a user might not personally like the design of a law firm (finding it too formal for their personal preferences) but it may still leave them with the right feeling about the company (that it is professional and trustworthy).

Second, when asked what they think users fallback on comments such as “I don’t like the green.” Comments like this aren’t particularly helpful. You need to know what it is about the green that is a problem.

Concerned about the effectiveness of your sites’ design? Have a client or boss who needs convincing? Get a website review.

Focus on brand value and personality

The most important thing to establish about a design is whether it is a good reflection of the company it represents. To do this you need a strong idea of what the characteristics of the brand is.

If you read the first post in this series, hopefully you will already have a strong vision of that personality. At the very least you will have a famous person you feel represents your brand, but with any luck you also have a list of keywords too.

This list of words (e.g. casual, energetic, passionate) are words you can test against. Does the design actually communicate a feeling of a passionate company among your users?

The question now becomes, how do you test a design against a set of words?

Techniques for testing design

What follows are the different testing techniques we have toyed with at Headscape. This is not a comprehensive list and some we use much more than others. However, it should give you an indication of your options.

Each test has its strengths and weaknesses, so a combination is often the way to go.

A/B tests

The most basic test is a simple A/B test. This is ideal when you are not sure of the best approach. Sometimes you can go different directions with a design and you are unsure which is best suited. In this case a simple A/B test maybe the answer.

Normally the A/B test simply involves presenting two designs to a user and asking which they prefer. This is perfectly adequate, but you might wish to go a step further.

Verify App A/B Testing is great for testing design
Verify App offers a great A/B testing service perfect for testing design.

Instead of asking them which they prefer, show them your list of brand words and ask which they feel best represents the list. This moves the conversation on from simple likes and dislikes, to something directly related to your brand.

Semantic differential survey

The semantic differential survey is a fancy name for a simple idea.

In this test you present the user with either single or multiple designs and rate them against opposing words.

For example is your design formal or casual, minimal or busy. These choices can be either a black and white choices or a sliding scale.

A Semantic differential survey
A semantic differential survey is a great way of seeing how your site is performing against your chosen keywords.

One tip if you go down the sliding scale route. Rate using an odd number of options (e.g. One to five instead of one to ten). If you pick an even number then users will often pick the middle options (e.g. five). An odd number forces users to lean one way or the other.

Competitor reaction

Another approach is to see how well your new design performs compared to the competition.

In this test you create a grid of your design next to the competitors. You then ask the user to pick which site best represents each of the keywords on your list (e.g. which is the most formal, reliable, professional).

Competitive word comparison test
Rating your sites design against the competition will give you an idea of whether you have successful differentiated yourself.

This is a great way of seeing whether you are out performing the competition. It also will flag up a common problem where designs within a sector all end up looking the same. If users struggle to select one site over another then it is obvious you have not done enough to differentiate yourself.

Snap test

The final test focuses on that first impression. We know that users make judgements about sites in a blink of an eye and so we need to accommodate this.

It has been shown that if people have a negative initial impression of a site they are more likely to look critically at it in terms of content and usability. It’s therefore important that the site makes a good initial impression.

The snap test is as it sounds. You show the user your design for only 1 or 2 seconds and then get their reaction.

Because of the short timeframe you may find that users struggle to judge the design against your list of keywords. However, you can at least discover whether they had a positive or negative reaction to what they have seen.

Aesthetics matter as much as usability

Many web designers like to refer to themselves as “user centric designers”. This term has put an emphasis on usability testing and rightly so. However, usability testing is far from the only form of user testing.

I would argue that it is just as important to get the aesthetics and brand values of your website right, as it is the usability. A site can be incredibly usable and yet totally forgettable. It is important that we test both areas.

In the next post I turn my attention to the usability side of design. We look at the tools and techniques available to help us get the structure and organisation of content correct.

“Scientist fingers holding a glass test tube in a research lab” image courtesy of Bigstock.com