As the saying goes, "You can't please all of the people, all of the time." So, how about pleasing 'most of the people, most of the time'? If you're an informational website owner aiming to improve session time, page views, bounce rate and ad revenue - then multivariate testing (MVT) is the biggest lever you can pull to accelerate your site's overall performance. The trouble is, even though the results are attractive, it can be a slightly daunting prospect.
Here are the top five concerns for most webmasters about multivariate testing (and some explanations to help put your mind at ease).
No. Informational publishers are now adopting this methodology to improve visitor experience and display ad revenue. Multivariate, or split testing, is the process used to test a hypothesis online. Usually, this is done by showing multiple page variants of the same content to separate user segments and comparing the results. Ecommerce sites do this all the time to improve sales. Amazon famously tested and retested their 'add to cart' button (orange works best, who knew?!).
Whether you designed your site from scratch, or chose a template you thought looked nice, there will be some personal bias built into the site's layout and 'look'. Some of this bias will get in the way of optimal user experience. The position of your site's menus, icons, content, ads and images all effect usability. While personalization of a site is great, the 'look' of a site is only a small part of what makes it unique. If you contemplate for a moment why your site gets traffic, it's because of your content, not its looks.
In the same way that an ecommerce site's purchase process is repeatedly optimized to give shoppers the most frictionless buying experience, a content site must deliver information to visitors in a seamless and intuitive way. Beauty is in the eye of the beholder, so you should let go of subjective attachment to what you think looks good and be open to testing change. If it's not to your personal taste, but your visitors like it more, then isn't that what matters most?
As the Guardian newspaper did when they announced their own mobile-site testing program, you may need to explain to your loyal visitors that site improvement is good for everyone in the long run: "By listening to our users and observing how you use the site, we improve incrementally. We release products and features bit by bit, learning as we go along. Some ideas make it through, some don't. The quicker we know what works and what doesn't, the quicker your experience will improve."
But let's face it, not everyone is going to be enamored with the changes you're making. People don't like change. So if you want to improve, you'll have to be prepared for some negativity from a vocal minority. However, if time-on-site and page views per user go up, and bounce rate goes down - that's positive feedback from the silent majority of visitors.
As an informational publisher, most of your income probably comes from display ad placements. It's the ads that keep the site going and give you an income. The questions are: Where should you place them? What about on mobile and tablet? How do you test placements? The answers could fill a whole separate article, but it's safe to say that testing new ad placements to improve your average income per thousand visits (EPMV) is a delicate business.
Some of your tests won't work and will generate less revenue than your original layout. Some will outperform for a short time and then drop back as more data comes in. However, when you get enough data and make adjustments, you begin to unlock your site's true potential to increase revenue and improve user experience. MVT can make sure ads are placed where visitors are most likely to see them, but also balance that with how long your visitors stick around and engage (the last part is crucial).
Testing is a continuous cycle of improvement based on visitor responses. When will Amazon stop testing their site? Probably never. How long does it take to see improved overall results (like increased time on site, page views per user and increased income)? It all depends on how much data you can generate and how quickly you can implement changes. If you have about 2,000 daily visitors, it will take about 1-2 weeks to get some statistically relevant results that add to your bottom line or overall user metrics. If you're constructing your own experiments, or have a smaller site, it will take longer than that.
Successful websites take a lot of time and effort to build, but it's worth it in the end. Multivariate testing is the next logical step for any publisher; it's using science to take what's already been created and make it better. The end result is that you can have a site that is improving all the time, making more money and is more appealing to your users too. What's not to like about that?
John Cole is Ezoic's Chief Customer Officer. Previously, John was a founding Director of Media Run Group, which started several online businesses in the mid 2000s. Following the acquisition of Media Run Ad Network by Adknowledge Inc. in 2007, John managed the Adknowledge Social Games division in Europe. After a successful foray into independent online publishing, John joined Ezoic in 2011 and moved from London UK to San Diego in 2013.