Imagine this: You have just finished leading the installation of a new testing system on the company’s marketing site. You’ve learned the new testing interface. You’ve done your research on A/B and multivariate testing and which to use when. All eyes are on you to improve revenue generated by the site.

Now you need to decide: What to test?

What are you testing for?

Before you start developing any tests, decide what you are testing for. Make sure you know the company’s goals. What is it that you are solving for by testing? What metric needs to improve for a test to be judged successful? Why does this metric matter? People get into trouble by not thinking this through before testing.

For instance, perhaps the VP directs you to increase the average time on page. You run some tests and successfully increase average time on page by 20%. Your site changes are judged a failure because even though you did what was asked – increase the time on page – the number of leads didn’t increase. The VP made an assumption that the longer a customer was on the page, the more likely he would be to submit his information. This assumption was wrong. What you should have been testing for was how to increase the number of leads.

To avoid this type of situation it is very important to understand what metric you are testing for and why.

Once you have your metric sorted out, the next step is to choose what to test. There are a lot of facets to deciding what to test, but to get started I’ll run through a couple of basic approaches: Best practice tests and audience motivation tests.

Best practice tests

Best practice tests involves fixing obvious usability issues. The goal is to get out of the way of your audience. It’s not the most exciting form of testing, but usually there are lots of areas to improve, especially if the site has never been tested before.

There are many ways you can find usability issues on the site. For instance, you can talk to the lead designer and see what she would like to improve. You can go through the site and ask yourself, ‘If I’d never seen the site before, what would confuse me?’ You can run usability testing and see where people hit snags.

This type of testing tends to be straightforward and generate good results.

Audience motivation tests

Audience motivation tests are more interesting, but typically harder to get right. While best practice tests are about getting out of your audience’s way, audience motivation tests are about lining the site up with what your audience wants to do.

To start thinking about audience motivation tests, consider these questions:

• What area are you testing? What is the metric?
• What does the company want the audience to do and why?
• What does the audience want to do and why?
• What do you want to the audience to do next?

The answers to these questions vary widely, but as an example:

• What area are you testing? What is the metric? The product pages. Revenue.
• What does the company want the audience to do and why? The company wants the audience to purchase their handmade artisan clothing to generate revenue and so forward the company’s mission statement of supporting local craftsmen.
• What does the audience want to do and why? The audience wants to purchase the clothing because it is attractive and distinctive, which feeds into their sense of self, and because they are emotionally committed to projects that help small businesses. These purchases also support how they want the world to be.
• What do you want to the audience to do next? To add a product to the cart.

If you have compelling answers for all of these questions your site changes are more likely to be successful. For instance, in this case perhaps you need to have better pictures of the product, because the audience is motivated by the clothing being distinctive and attractive. Or perhaps you need to include information about the person who made the clothing because this will reinforce the customer’s ideas about helping small businesses and about how they want the world to work.

Audience motivation testing is generally more difficult. It requires research and imagination. But, in the long term, you will build an understanding of your audience and their motivations, which will help you construct a more effective site. After all, without an audience, your site is just a collection of code.