Four Ways To Test A Web Page

A very simple way to quickly improve a website’s performance is to pick out some key pages (eg. checkout pages, signup forms, product pages) and try out different versions of the page to see which performs best. This article explains 4 different test methods & the advantages of each…

What Is A/B Testing?

An A/B test involves making two versions of a page (funnily enough, versions ‘A’ & ‘B’) with one single difference. Here are a few ideas for things to test:

• Price
• Image size
• Headline
• Text size
• Button/link placement

To run an A/B test, set up these two versions of a page & have them display in roughly equal amounts to your visitors. At the end of the test you count up which version gave you better results & then you stick with that version.

It’s worth figuring out a hypothesis beforehand. For example - if you’re going to test 2 headlines, you might think “With our target audience, a headline leading with the benefits is going to outperform a headline talking about price”. At the end of the test you should then be able to say “according to this test, benefit-led headlines perform better than price-led”. This is a simplistic example, but the idea is to gather knowledge that you can use again in the future - not just one-off improvements.

What Is An A/B/A Test?

An A/B/A test is where you split the test in three. You pick a variable & create THREE versions of a page. Two versions that are totally identical - “A” & “A(2)”. One version where the single variable is changed - “B”. You then test the three pages against each other.

It sounds counterintuitive at first: why would you compare one version of a page against the same version of a page? This might help explain it:

The answer is: with smaller sample sizes, there’s always going to be a ‘random factor’. Just here you can see that ‘version A’ beat version B by a mile. BUT when you look at the /other/ version A, the results are different. What happened here? Well, the first “A” just got lucky. Because this was such a small sample size, we didn’t get enough data to accurately judge which version was better. However - if the results had come out “A: 24″, “B: 13″, “A(2):24″, we could have been relatively sure that version “A” was genuinely the better performer.

In short - A/B/A testing adds an extra safeguard on your results & gives an indicator of how sure you can be of their significance.

What Is A/B/n Testing?

A/B/n testing is just the same as ‘A/B’, only with many more variations. Say you’re testing to see which headline will drive more sales on a page, & you know you can get 80,000 visitors to that page, that’s a huge amount of data to waste on just two different versions of the headline. So you set up 10 different versions, eg:

A: Buy Shoes Now
B: The Shoes That Will Improve Your Performance
C: Today Only - Super-Hot Shoe Bargain
D:
…

And then measure the results for each & stick with the better performer.

What’s so good about this? It saves you time in the test process &, if you have a system to automate this, it allows you to just plug in a whole series of ideas & gather a lot of feedback quickly.

Bonus Tip: Throwing in several similar variations can tell you a lot. For example - let’s say you have a hunch that price is going to be a decisive factor. Testing ‘Buy Now - Only $65′ vs ‘Buy Now’ would help you, but testing those two alongside ‘Buy Now - Under $70′ & ‘Buy Now - Excellent Price’ would tell you a lot more.

What Is Multivariate Testing?

Multivariate testing is basically where you run a whole series of A/B/n tests on a single page all at once. It’s where you test /many variables/ on a page at the same time. Here’s an example - let’s say you’re testing 3 different variables & you have 3 versions of each, eg:
Headline

• Header A: Buy Shoes Now
• Header B: Incredible Shoes
• Header C: Ridiculous Shoes

Price

• Price 1: $70
• Price 2: $68
• Price 3: $78

Text Size

• Text Size i: 12px
• Text Size ii: 14px
• Text Size iii: 9px

That actually gives you (if my maths is right) 27 different versions of the page. You may think “But if I’m testing all those variables at once, how do I figure out what’s causing the increased/decreased success of the page?” - that is where the ‘traffic volume’ factor & some clever automated analysis comes in. If you /always/ get better results with Text Size iii, then it’s fairly safe to say that’s a winner.

Unless you have a lot of time on your hands, multivariate testing only makes sense if you use an automated tool (eg. Google’s Website Optimizer) to measure it & act on the results. But, if you do get a decent amount of traffic & you can automate this, it’s is a great way to use the behaviour of your visitors to improve your results very quickly & automatically fine-tune your site.

Technorati Tags: web analytics, a/b testing, conversion, multivariate testing