Measure Your Site Changes With Visitor Snapshots

If you’ve ever made regular, site-wide improvements to a website you’ve probably come across the problem of trying to please all of your visitors all of the time. You fix something because a couple of loyal buyers complain about it, then you notice your ‘new customers’ number is dropping. You simplify the cart to try and fix that, and suddenly your loyal buyers are complaining again. You look at your sales numbers & everything looks fine. Where did you go wrong? Are all of your ‘loyal’ customers unhappy, or is it just a few shouty complainers?

After a while this causes problems & slowdowns on new projects:

• Watered-down projects that attempt to please everybody & fail to please anybody
• Frozen action - you’re not sure what the fallout will be, so you just don’t do anything
• Your internal team disagrees over how to tackle a problem & you just end up arguing

So what do you do?

The simplest answer is to test it. Build a simple snapshot scorecard allowing you to take a few ‘before’ & ‘after’ pictures to show how the change has affected your important visitor groups.

Example 1: A Basic E-Commerce Site

Here’s an example snapshot of a few key metrics from an e-commerce site. The metrics (across the top) could be anything important to your store. The visitor types (down the side) could be unique to you too: 5-time-plus-buyers; buyers by vertical; visitors from search engines; etc.

Before snapshot

Here we can see that 67% of ‘previous visitors’ (people who have come to the site before) reached the ‘product browsing’ stage (ie. they looked at a product page). 18% of them added something to their cart & 2.3% of them purchased something.

After snapshot

A week after you implemented the change, you take another snapshot. By looking at the numbers that have changed, you can see exactly who has been affected, how they’ve been affected & whether there have been knock-ons elsewhere. Here we can see that our sales to ‘new visitors’ have jumped up from 1.5% to 2%. But, sales to ‘previous visitors’ have dropped by a full percentage point to 1.3%. The topline sales figures might have hidden this, but the snapshots allow us to get closer to the real picture.

Example 2: A Blog Containing AdSense

This technique isn’t just for e-commerce sites. It can work anywhere where you can define measures of success (about 99.99% of websites). Here’s an example of what you might look at on a blog:

Let’s say you change your blog’s theme. That’s likely to have a huge impact - but how do you measure it? Something like this gives you a quick visual representation of exactly what’s going on & what it’s likely to mean in the short-term (new visitor adsense clicks) & in the long-term (previous commenters trackbacks).

Expansion

Once you’ve set something like this up it makes a great addition to your monthly (or weekly) scorecard to keep track of movements up and down. This helps to identify problems, report on success & measure the effects of any site/audience/search ranking changes over time.

Another good expansion of the idea is to do A/B rather than before/after testing. If you can run an A/B test on your major site changes, using snapshots like this for each of the groups, you can then fine-tune your site changes until all of the numbers are going in the right direction & you’re happy to roll it live to your entire audience.