Before we get to the meat and bones of this post lets first consider the landscape prior to Google’s free Website Optimiser tool. Historically, A/B and multivariate testing platforms were expensive, a luxury only available to those with deep pockets. These same people could afford the traffic needed to run AND complete complex tests with conclusive winners within business effective time frames.
Fast forward to today and we have Google’s Website Optimiser, a free, very capable tool that has effectively made A/B and Multivariate testing available to the masses. For this I applaud Google, they do of course have their own motives but even so it’s a great tool and everyone likes a Google freebie!
The Problem
Google have a habit of over simplifying, they’ll promote the Website Optimiser as a free easy to use system that anyone can pick and use. In fact, you’ve only got to look at the language on the Website Optimiser site to see what I mean. According to Google, Its “great” for “first time testers” who can “quickly” set up a test in “five minutes”. This unsurprisingly encourages newbie’s to jump into testing feet first with little or no thought regarding what they are testing and more importantly WHY they are testing it.
This problem is compounded by a general lack of quality information/guidance and the fact that a lot of the people claiming to be “experts” out there are waxing lyrical recycling what they’ve heard from a few genuine experts with little or no experience of their own.
Now I’ve stopped moaning onto some tips:-
Tips to Get You Started
Plan properly: I tend to apply a 60/40 rule to tests. In that, 60% of my total time allocated is spent analysing data, thinking, considering, planning what I’m going to test with 40% of my time allocated to setting up the test and combinations. Don’t run a test for the sake of running a test; it’s likely to be a flop if you do that.
Do your maths: make sure you’ve calculated how many combinations you should be able to run in a reasonable time period (max 4 weeks). Most people want results quickly, the fact is few businesses can afford to or are prepared to wait 6-8+ weeks for a test to be conclusive. Getting the test through to a successful conclusive is your number one objective. There are a number of online calculators available, first calculate how many combinations you think you can run in your desired time frame and then halve it.
Set the right goal: Make sure you are measuring success in the most relevant way possible. This should be considered during your planning stage. For example, if you are testing an important landing page with a high bounce rate and your objective is to reduce this bounce rate any click to a second page is a goal. In fact, why not track multiple goals using Google Analytics so you’ll understand the impact of each page version on bounce rates AND revenue.
Be Bold: Small Changes = Small Results. Make sure your new versions are significantly different to your original. This is my pet hate and the reason so many tests drag on and on. Unless, you have zillions of visitors/conversion don’t waste your time testing 500 versions of a button with each slightly different to the original. Again, this should all be considered during the planning phase. A carefully considered test with a handful of very different versions will yield bigger, faster results.
That’s it for today; I’ll be adding more tip and pointers as I think of them and covering each of the above points in detail individually at a later date. Even so, actioning the above pointers today will increase your chances of producing successful, conclusive tests that don’t take decades to complete!
