Are you doing A/B testing?

Its easy, its cheap and it could increase the conversion rate on your website significantly. Why then are we not all doing A/B testing?

I seem to keep coming across articles and applications relating to A/B testing recently. For example somebody in the Boagworld Forum mentioned ABtests.com, a brilliant site for reviewing other people’s tests and the benefits they provided.

ABtests.com

I was also reading an interview with Tim Ferris in which he talked about the benefits of A/B testing. He went on to talk about how to avoid being overwhelmed with too much data.

“Dont measure too many things,” he says. “People often become overwhelmed with a deluge of data because they’re looking at 1,500 variables. And that can be paralyzing because you end up sitting there looking at your analytics program all day long.”

Good advice.

All of this made me feel kind of guilty because although I know A/B testing is incredibly beneficial I rarely do it on anything other than client websites. This is particularly stupid when you consider that there are some excellent free tools available for A/B testing. Take for example Google website optimiser. It is entirely free and relatively simple to set up if you know how to copy and paste code into HTML.

Google Website Optimizer

Even if you don’t know how to edit HTML there is still nothing to stop you doing some A/B testing. There is a tool called visual website optimiser makes the process even easier and provides an WYSIWYG environment for creating your alternate versions.

visual website optimiser

Why is it then that so many of us do not bother A/B testing websites? I would be interested to hear your thoughts in the comments.

You're a failure: Deal with it

Every website has points of failure. It is inevitable. The question is do you know what they are and are you doing something about them?

Nobody likes to think of themselves as a failure and no website owner likes to dwell too much on the shortcomings of his website. However, all websites have weaknesses and it is important we know exactly what they are in order to do something about them.

What is more, all websites develop new weaknesses over time. As content is added, structure is changed and designs are tweaked, we introduce problems into our sites that were not previously there.

We therefore need a system in place that continually monitor for failures so that they can be addressed quickly before causing too much damage.

image of broken chain

Torian, Shutterstock

Implementing such a system does not need to be onerous. It simply consists of three simple reviews that should take place on a monthly basis. These are…

  • Monthly user testing
  • Identifying dropout points
  • Analyse search queries

Let’s begin with user testing.

Monthly user testing

Steve Krug’s latest book Rocket Surgery Made Easy takes usability testing in a radical new direction that is both perfect for identifying the constantly evolving weaknesses in our sites and also inexpensive to implement.

He proposes an ongoing programme of quick and dirty user testing that takes place every month. The idea is that on a set date each month you schedule a morning of user testing with only 3 participants.

Your entire web team watches the user testing and analyses the results over lunch. By the end of lunch you will have identified the biggest problems that need to be addressed before the next month’s testing.

Image of Rocket Surgery Made Easy

Jason Alley, Flickr

This is a great approach for detecting and eliminating problems on your website. It is…

  • Lightweight – Only requiring one morning a month for testing and debrief.
  • Regular – Ensuring that it picks up on problems that creep into the site over time.
  • Action orientated – By the end of the session you have a plan of how to tackle the failings in your website.
  • Fixed – Because the user testing is always on the same day every month it is less likely to be bumped for more ‘important‘ things.
  • Prioritised – With only a short time to test and debrief the emphasis is placed on the most important failings rather than being caught up in nuances.

Although user testing is useful it should not be used in isolation. Testing only 3 participants enables lightweight and regular testing, but it does have its limitations. Ultimately 3 people are not going to be representative of the whole and even if they were, they are not interacting naturally with the website. For that you need to look at web statistics.

Identifying drop out points

Website statistics can be very enlightening when used correctly. Unfortunately few people know what they are looking for and so either give up try or never get past page views and unique visitors.

One aspect of web stats that is particularly interesting is exit points. Where do users leave your site? This may help identify potential points of weakness in the site and areas you wish to test in your next user test session.

Most statistic packages make it easy to view a list of top exit pages. However make sure you are viewing pages with the top percentage of exits, because popular pages will be exited more often. Look for pages that are popular but are also exit points. These are the biggest problem areas. Google Analytics actually allows you to view pages that meet this exact criteria.

Google Analytics - Top exit pages

Once you have this list ask yourself the following questions?

  • Is this the final page in a call to action process? For example is this an order confirmation at the end of a purchase process. You would expect pages such as this to be a typical exit point.
  • Has the user visited any other pages before exiting? If the user has viewed only this one page then the chances are they were simply at the wrong site. To make sure check the dwell time. If they were only on the page a few seconds before leaving then it is fair to conclude they had arrived by accident. Also check how they arrived on the site. If they came via a search engine, what search term did they use? If the term wasn’t relevant to your site then don’t worry.
  • What type of content does the page contain? If the page is a blog post for example the chances are the user was more interested in the content than anything else you offer. They will read the one article and then leave. Obviously this is not what you want and should work hard to encourage them deeper into the site. However, the reality is that pages of this type will have a higher exit rate.
  • Is the bounce rate significantly higher than elsewhere on the site? If so this could indicate a weakness in the page.

If users have viewed multiple pages and then given up on a particular page, it is a problem which needs addressing. Possible problems could include…

  • No obvious next steps – Are you telling the user what to do next.
  • Too many optionsToo many choices can cause users to give up.
  • The content fails to convince – Is the content of the page telling the user what they want to hear, or are you saying something that alienates them?
  • Too much content – Is the page packed with large amounts of densely written copy. This can put users off.
  • The content is not relevant – They have arrived from a search engine query and the page hasn’t provided what they want. Instead of looking elsewhere in the site they have returned to the search engine to view another result.

Unfortunately although web stats can be very good at identifying problem pages it is not so good at diagnosing the cause. That is why it is important to user test as well.

Of course another possibility is that a user has given up simply because the site doesn’t appear to have what they are looking for. That is where you need to analyse search queries.

Analyse search queries

What users searched for provides an excellent insight into potential failures of your website.

Take for example the top search queries that lead people to a page with an exceptionally high bounce rate. Which of those queries cause most of the bounces? What are the user expecting to find on the page and do not? The page must be relevant in someway otherwise the search engine wouldn’t refer them. However, perhaps the way you are presenting the information is wrong. Does the search term give any indication of how you could be presenting things better?

What about the terms that create the highest bounce across the site. Sometimes users find themselves on a site that a search engine believes is relevant but doesn’t directly address their issue. Could you retailor your content to more directly address these search queries?

Keyword search terms as shown in Google Analytics

However it is not just external search engines you need to be looking at. What about your internal search engine?

When a user arrives at your site after entering a search term into Google, you can be sure that at least some content on that subject exists. Otherwise Google would not have referred them. However, when a user types something into your site search, there is no guarantee it will be a topic you have addressed at all. This is a perfect way to identify content lacking from your website.

Repeat each of the top searches yourself and look at the results. Are you addressing the search terms directly? Are you addressing them at all! If not you have identified a weakness that needs dealing with.

Rinse and repeat

The secret to success when it comes to irradiating the failures on your site is to establish a monthly cycle of work. Each month you need to user test, delve into your web stats and analyse your search results. By combining these three techniques you should be able to establish a programme of work for the coming month. By repeating this process month on month you can slowly evolve your site so that its shortcomings have less and less of an impact on conversion.

Of course, this is not all that can be done to identify problems with your site. These are simply those that have worked best for me. What about you? How do you find the shortcomings on your site? Do you even do this kind of ongoing maintenance? Let me know in the comments below.

193. Get more from Google Analytics

On this week’s show: Paul and Marcus are joined by Matt Curry who shares some advanced Google Analytics techniques. We have a review of Fancy Form Design by Jina Bolton and Paul goes on endlessly about the Website Owners Manual.

Play

Download this show.

Launch our podcast player

Housekeeping

How can I not mention the launch of my book the Website Owners Manual? You are going to be sick of hearing about this, but console yourself with the fact that I have a very short attention span and will soon get bored of it. Please take a few minutes to learn more about this book at boagworld.com/websiteownersmanual. I would especially encourage those of you who are web designers to check it out. This book contains all the information your clients ‘need to know’. It was written specifically to be given away to clients, so helping your projects run smoother. I even managed to pursued my publisher to give significant discounts to those buying more than 5 copies. However, as an extra bonus for boagworld listeners you can also get an additional 40% off of any website owners manual purchase (including the multi-buy packs) if you use the code ‘boagworld’ at checkout.

Back to top

News

Design interactive prototypes – Fast!

With websites and web applications becoming increasingly complex it is often hard to visualise them before build. Photoshops comps fail miserably and static wireframes are little better. The only way of truly communicating how a site is going to work is to build an interactive prototype. Unfortunately building prototypes can be time consuming and expensive. Although clients need to understand how their site will work, they are rarely willing to pay for a prototype. One solution is IxEdit, an ‘interaction design tool.’ This tool has to be seen to be believed, but essentially allows designers to build jQuery driven prototypes without writing a single line of code.

With IxEdit you can build everything from the automatic insertion of HTML to accordion effects. In fact you seem to be able to build most of the elements and effects supported by jQuery. Of course the quality of code is not going to be as good as something written by hand. That is why the product is billed as ideal for prototyping. However, for better or worse, I am sure a lot of web designers will use this tool for live sites too.

Making passwords more usable?

On the subject of Javascript and interaction, there is some interesting work being done with password masking. In show 173 I talked about some of the problems surrounding password masking. Essentially, although hiding passwords increases security it also creates a usability challenge. Jakob Nielsen wrote:

Usability suffers when users type in passwords and the only feedback they get is a row of bullets. [It] costs you business due to login failures. Password masking has become common for no reasons other than (a) it’s easy to do, and (b) it was the default in the Web’s early days.

There have been a few solutions doing the rounds. The simplest of which is to add a checkbox allowing users to keep their password entry hidden. However another popular approach is the one adopted by the iPhone. Instead of revealing the entire password it shows only the last letter entered. These two approaches have now been combined and made simple to implement using a sprinkling of jQuery. Delayed Password Masking couldn’t be easier to setup and helps go someway to improving usability.

How to be more transparent

In my post “The 10 Harsh Truths About Corporate Blogging” I wrote:

People don’t like interacting with organisations, corporations or machines. People like conversing with people. People don’t like, trust or want to work with corporations. We associated those feelings with individuals, not companies.

In other words, if you want to make a connection with your users you need to be open, transparent and show the people within your organisation. However, knowing this and doing it, are two different things. That is where a recent UX Booth post comes in. The title of the post is “Transparency: Benefits and Best Practice.” Personally, I think this is a misleading title. It doesn’t really explain in any depth why transparency is important and fails to provide much in the way of ‘best practice’ (I can see I will have to write something on this subject). What the post does do well is give you some cracking examples of sites that communicate the personalities and people behind their organisations. It certainly has inspired me to look again at the Headscape website, and I hope it will inspire you to become more open as an organisation.

In other news – Google and Microsoft talk about stuff

Normally I like to keep the content of this section of the show focused on the here and now. I see little point in reporting what might affect you ‘one day’ in the future. That said, there are two stories that have come out this week, which I simply couldn’t ignore despite the fact neither will have an impact on you today.

Google to add site speed to search algorithm

This week when talking about the importance of website speed Matt Cutts from Google said:

Historically, we haven’t had to use it in our search rankings, but a lot of people within Google think that the Web should be fast. It should be a good experience, and so it’s sort of fair to say that if you’re a fast site, maybe you should get a little bit of a bonus. If you really have an awfully slow site, then maybe users don’t want that as much.

If Google follow through on this thinking the consequences could be massive. In particular this could further undermine the already shaky rankings of flash heavy websites. It could also provide a real advantage to those with the financial resources to throw more server and bandwidth capabilities at slow websites. That said, on the upside it would refocus website owners on the importance of performance and help to speed up the web for everybody. It will also encourage better coding practices maybe push legacy tables based websites down the rankings. Of course all of this could be redundant. We have no way of knowing whether Google will implement this change, and even if they do, how great a priority they will place on speed.

Microsoft talks about IE9

The other news that might shape the future of the web comes from Microsoft. With Windows 7 complete it would seem they are turning their attention to Internet Explorer 9. Apparently the new browser is only in very early stages of development. However, Microsoft are making it clear what their priorities for the browser are. These include:

  • A desire to provide better HTML5 support
  • Significant speed increases for Javascript
  • Improved CSS support
  • Better use of hardware acceleration

All music to my ears. However, I was sad to read that according to Mashable they have only been working on the new browser for 3 weeks!

Back to top

Interview: Matt Curry on Getting more from Google Analytics

Transcription to follow shortly.
In the meantime follow Matt on Twitter.

Listeners book review: Fancy Form Design by Jina Bolton

What is it?

This book, in Jina’s own words, is aimed at anyone who’s involved with any part of the creation of an online form. Split into 5 sections, it covers the Planning, Designing, Structure, Styling and Enhancing of forms used on the internet Written in a format that is more about advising and guiding rather than teaching, this book will appeal to people who are used to the Sitepoint way of writing, and want to really understand the thinking behind creating a successful web form. It’s not one of those “learn in 24 hour” type books, but is more written as if you’re at a workshop run by Jina. This is not a hardcore reference manual that covers absolutely every permeation of a web form, but will have you more confident and eager to apply what you learn to forms you build from now on.

No bloat

With this book, Jina has tackled a subject that frustrates many a web designer. Forms are often too time consuming, too technical, or too stubborn to spend time getting right. Resources on the internet fall usually into 2 categories, not enough info, or too bloated and confusing. What Jina has managed to do is get straight to the point, without the bloat.

A form is just a form. Isn’t it?

Straight from the 1st chapter Jina had me thinking differently about forms. Before reading this book, I would not have said things like sliders, colour pickers, or drag and drop items are elements of form design, but when you look at where they are used, it’s obvious they are. I’m already more excited about forms than I was before. And I think that’s what this book does really well. It takes the process of form creation, and says “yeah, I know, a form is a form. But look, you can do this with it…”. Jina shows you how a form is very much like a website design. You need to think about typography used, colours & imagery, how the form is going to be structured and how it will affect how it used.

Good practices make perfect

Throughout the book, Jina runs through some processes for creating perfect forms. It starts with how to research and find inspiration. Many people who have built forms in the past would probably not have used the processes talked about in the book. It’s an eye-opener to best practice, and to how investing time in tried and tested techniques at the beginning will save you time further down the line. Many of the practices Jina talks about are transferable techniques, that can be adapted and implemented on web design, brochure design, database design etc. What I really liked is the way the book doesn’t force you to follow the practices, but is more like a friend giving you some tips.

Get your hands dirty

Although I mentioned this book isn’t a “teach yourself in 24 hours” jobby, it is by no means a pure reference book. You can follow along with Jina, and get your hands dirty with some HTML markup and CSS. JavaScript is kept to a minimum by using jQuery, and again has example code you can work along to.

In a nutshell

Fancy Form Design is probably the best title for this book. It explains how to design forms that look fancy. Jina does not pretend this book will make you a master of AJAX form submission techniques, nor an expert in JavaScript server-side form validation. It breaks down the components of creating a form, the content of that form, how to jazz it up with some clever styling tricks and jQuery magic, and makes you think about forms more as an important part of your design rather than a stone in your shoe. To me, this book does exactly what it says on the tin. Buy Fancy Form Design from Amazon

Back to top