The whole topic just really confuses me.

Where I am most puzzled at the lack of A/B testing is among small business owners that use websites to sell their products. In these cases there is a clear conversion criterion (customer adds something to their shopping cart) and a strong motive to increase the conversion rate! Yet no-one seems to be using A/B testing.

My own theory is that while WordPress, Ebay, Etsy, Shopify etc. has made it very easy for a small online business to get up and running, A/B testing tools still require a fair degree of technical know-how. Even stuff like Visual Website Optimizer requires you to insert Javascript into your web pages. There are plenty of small online business owners who would have no idea how to do this!

I think that the key to getting people to start using A/B testing (and I admit I am strongly biased; I run an A/B image testing webapp) is to (1) make it easier to get started and (2) concentrate on the benefits of optimizing a single aspect of their site at at time. I concentrate on talking about optimizing product images, since I think this is an easy and intuitive way to get started. In other words, once you show people the benefit of A/B testing with something easy then they will motivate themselves to learn how to use more complicated tools like GWO.

Why we don’t A/B test is a mixture of a lack of familiarity, a bit of pride (no need to test, my design MUST be the best!), and a lack of ownership over conversion rates. Most of the time we don’t measure what we do in terms of performance metrics, let alone try and improve it. Easier to fly blind, get sign off, get paid, and move on to the next project, I guess.

So, it may seem round-about, but until we take performance and web analytics seriously — i.e., have a solid foundation for measuring what we do — I don’t think we’ll be that inclined to do much A/B testing. I intend on doing my bit to change that ;)