How to Define Your Digital Success and Track Usability

In this lesson we will explore:

• Why you need to define success.
• How to define key performance indicators.
• What areas you should be assessing.
• How you should assess usability.

Let us begin by asking why defining success is so important.

Why You need to Define Success

No digital professional can do their job without knowing what they are trying to achieve. Whether you are a digital marketer, UX designer, developer, copywriter or any other role, you cannot see whether you have done your job well unless you have some way to measure success.

Not only that, having a definition of success focuses everybody on work that creates a tangible return on investment. Too often, site improvements are vanity projects, rather than alterations aimed at enhancing site effectiveness.

The focus that comes from having measurable success criteria has another benefit too. It helps to resolve disagreements between stakeholders over the best direction to pursue. When everybody has a clear goal, working towards it helps to ensure everybody is pulling in a consistent direction.

But a way of measuring success provides benefits to us as digital professionals too. It helps us to demonstrate the value of the work we do by enabling us to show how it benefits the business as a whole. That, in turn, also encourages further investment in digital, allowing us to be more ambitious in the work we undertake.

The question then becomes; how do we go about defining success?

Defining Success Through Key Performance Indicators

Organisations typically measure success by defining a set of key performance indicators. These are quantifiable measures to follow when performing a site audit to evaluate the progress of your website over time.

However, deciding what these metrics should be is not always easy.

1. Start With Your Company Strategy

A good starting point is the overall company strategy. Most organisations have a plan that outlines broad goals they are looking to achieve. That might be increasing revenue, decreasing costs or extending market share.

2. Transform Strategy into Project Goals

Once you have these broad organisation goals, seek to identify ways that a high performing website could help to achieve them. That might be generating more leads, reducing support queries or increasing the number of shares on social media.

3. Turn Your Goals Into Measurable Criteria

Finally, turn these criteria into something measurable. For example, generating more leads might be measured by the number of contact forms submitted, while reducing support queries might be a drop in the number of people phoning the company.

Of course, no metric is going to be perfect. Some people will email instead of completing a contact form, and there is no guarantee that the drop in calls is exclusively due to the website. That is okay. Make your metrics as good as possible, but accept they will never be perfect. It is better to measure something than nothing.

However, let me share a word of warning. Don’t become too obsessed with your metrics as they are often less than perfect. They can only ever be an indicator of success.

Also, if you settle on the wrong metrics, it can damage the sites overall performance. For example, if you focus purely on lead generation results in annoying popup overlays that will hurt long-term customer satisfaction and sales.

To avoid these problems seek to have a mix of metrics that balance one another.

Striking a Balance in Your Key Performance Indicators

Typically I attempt to have a mix of three areas to ensure a balance. These are:

• Engagement,
• Conversion,
• Usability.

Let’s take a brief look at engagement and conversion, before focusing in on usability, as that is the main topic of this course.

We will start with engagement.

Tracking Engagement

Ensuring a site is usable should be a minimum expectation. In truth, our sites should be doing so much more. They should also be delighting and engaging users. We can measure engagement by tracking metrics such as:

• Attention minutes. How long a user is paying attention to content such as videos.
• First Impressions. How users initially react upon seeing the site for the first time.
• Interactions. How often does the user like, comment on or share content?
• The Net Promoter Score. A metric for measuring how likely a user would be to recommend the site to others.
• Interaction depth. How many times does the user click when navigating through the site.

Tracking engagement often aligns with organizational goals around increasing brand awareness, expanding market share or other marketing orientated metrics. However, be careful not to focus on elements such as visits because factors beyond the website can significantly influence these figures.

Tracking Conversion

While as UX professionals, we focus on engagement and usability, management and colleagues are probably going to be more concerned with conversion rates. Ultimately, the majority of websites exist to persuade users to take some form of action. These actions often create a tangible financial benefit for the company.

If the website belongs to a retailer or service provider, it is typically relatively easy to track conversion. E-commerce sites can track sales, while service companies track leads generated.

However, things are not always straightforward. For example, tracking the number of leads generated is not necessarily the best metric if those are not qualified leads. Also, it can sometimes be hard to identify whether they have come from the website, rather than through other means.

It is in these kinds of situations it is easy to give up on the idea of tracking metrics. However, as I said earlier, it is better to monitor something even if it is not 100% accurate. Also, with a little imagination, it is often possible to track more than you think.

Finally, where possible, it is also good to associate financial value with these conversions. For example, if the business knows that approximately one in 10 of the leads generated on the website turns into a paid project and that the average project is worth a particular figure, then it is possible to work out how much each lead is worth to the business. This kind of calculation is worth doing as it helps justify further investment in the site.

Now that conversion and engagement are out of the way, let’s focus in on usability.

Measuring Usability: What Metrics Should You Track?

Measuring usability is often overlooked in favour of a focus on conversion. However, ignoring usability as a metric can have a significant impact on the long-term health of your business and ultimately will undermine users taking action.

Why You Should Be Measuring Usability

Usability underpins everything on a website. A website can be unattractive, un-engaging and even poorly written and it may still convert. However, if people cannot use a site, taking action becomes impossible.

Worst still, users have an extremely low frustration threshold when using a site. With so many competitors out there, why would a user struggle with a problematic website?

But good usability can provide a distinct business advantage too.

There is a reason why every Christmas I say to friends and families that they can have whatever they want as a gift, as long as I can order it from Amazon. I know Amazon, and so for me, it is easy to use.

My familiarity with Amazon makes me loyal to that site, and I am not alone. Users tend to be loyal to sites they find easy to use. That increases the customers lifetime value, allowing companies to pay more for customer acquisition so outperforming competitors in pay-per-click listings.

Customers will even pay a premium for a straighforward experience. That means if your website is easy to use you can potentially charge more!

How then do you know if your site is easy to use? How do you measure its usability?

Measure Your Site’s Overall Usability

There are many ways to measure a site’s usability. However, most of the options are relatively time-consuming to implement, which means that they do not happen regularly. That makes it hard to track the effectiveness of site improvements over time.

Pragmatically most organisations need a cheap, quick method of measuring usability and that is where the system usability scale comes in.

The System Usability Scale

The system usability scale is a simple survey that asks users to express their agreement with a series of statements. Participants rank each statement from one to five based on how much they agree or disagree with it. Five means they completely agree, while one means they strongly disagree.

What statements you use is of course, entirely up to you, but ideally, you should have ten. A typical sample list of statements for a website might read:

• I think that I would like to use this website frequently.
• I found the website unnecessarily complex.
• I thought the website was easy to use.
• I think that I would need the support of a technical person to be able to use this website.
• I found the various functions in this website were well integrated.
• I thought there was too much inconsistency in this website.
• I would imagine that most people would learn to use this website very quickly.
• I found the website very cumbersome to use.
• I felt very confident using the website.
• I needed to learn a lot of things before I could get going with this website.

How to Calculate Your System Usability Scale

Notice how the above statements alternate between positive and negative statements. For example, the first statement is positive:

I think that I would like to use this website frequently.

While the second is negative:

I found the website unnecessarily complex.

For each positive statement take the users score and subtract one. So, a score of four would become three.

For each negative statement subtract the users score from five. So, if a user scored a statement as a five, then the final score would be zero.

Once you have a number for each statements, add these together and times the total by 2.5. That will give you a score out of 100.

Admittedly all of that mathematical gymnastics is a bit fiddly, but it does give you an easy rating out of 100 for measuring usability. A metric that allows you to track improvements over time.

How to Use the System Usability Scale

Once you have the hang of it, the system usability scale opens up a world of possibilities.

For a start, if you use the standard statements as outlined above, you can get a reasonable idea of how usable your site is compared to other systems in general.

The average system usability score is 68 and anything above 80 means you are in a very good place. However, anything under approximately 50 and you need to be prioritising usability fixes.

But, you can also use the system usability scale to compare your site to the competition by asking users to rank multiple sites.

Finally, you can use the system usability scale to rate a prototype against an existing site or to compare multiple design approaches.

The Limitations of The System Usability Score

Although the system usability scale is an excellent way of measuring a site’s usability and by extension the impact of usability on conversion, it is not perfect.

The system usability scale suffers from two flaws.

First, it can only identify how good or bad the usability of a website is. It cannot diagnose why it is succeeding or failing.

Second, there is often a discrepancy between what a user says and reality. For example, somebody may describe a website as easy to use because they are familiar with it. Equally, they might say they will recommend that site to others in the moment of completing a survey, but not do it in reality.

These facts mean that we cannot only rely on the system usability scale. We also need to be measuring how users perform when completing critical tasks on a website.

How to Measure Usability of Critical Tasks

If you want users to complete a call to action on your website, taking that action has to be easy. But how do you know if signing up for your newsletter, buying a product or completing your contact form is, in fact, easy?

What Task Metrics to Measure

For that kind of specific action, the system usability scale will not help. We will need a different type of usability metric tied specifically to our calls to action. We will want to measure a series of factors. Factors such as:

• The time it took the user to complete the action.
• The number of users who simply failed to complete the task.
• Whether the user was able to complete the call to action using the most direct route.
• The average number of mistakes the user made when trying to complete the task.

Fortunately, these various metrics do not all have to be measured separately. We can ascertain all of those metrics by watching real users complete tasks.

How to Measure Task Metrics

You can track task metrics when running any facilitated usability test session as long as users are being asked to complete the appropriate tasks. However, because we are looking to gather quantitative data, testing with a more significant number of users is preferable as this reduces any one user skewing the metric with abnormal behaviour.

Because facilitated usability testing can be time-consuming, it may be preferable to carry out unfacilitated testing when seeking to establish metrics around critical tasks such as your calls to action.

In unfacilitated testing, the user is set a task to complete, and they then finish it without being directly observed or interacted with by a facilitator.

There are a number of tools that can help with unfacilitated testing of this nature.

One option is to use an application such as Lookback. Lookback allows you to carry out both facilitated and unfacilitated testing both in-person and remotely.

Lookback will allow you to send a link to a user which when clicked will set that user a task and ask them to complete the work while being recorded. The advantage of this approach is that you can see users completing the task in video format along with a commentary of them explaining what they are doing and thinking.

The drawback, however, is that you are left to calculate the metrics by watching each session and recording the number of misclicks or whatever other metrics that are of interest.

One solution to this problem is to use a tool like Maze. Maze doesn’t record videos of user sessions or allow you to see the user completing the task. However, it does provide detailed analytics in regards to what those users did. That makes it ideal for task metrics, while Lookback is better suited to qualitative testing.

Making Sure Measuring Usability Happens

Using the right tool for the job is of crucial importance if you are to integrate tracking usability metrics into your workflow. If gathering those metrics is not straightforward then you will stop doing it, and that is a dangerous road.

Our tendency is to focus on things we can see. If all we see are conversion numbers, then this will be the focus of the organisation. The result will be the adoption of short term techniques to improve conversion (dark patterns) at the cost of long term success.

If we start tracking usability, we not only ensure it gets the attention it deserves; over time we will also be able to demonstrate its value. We will be able to show that if we reduce the time it takes to complete a task, then there is a direct correlation with improvements in conversion.