I often hear heated arguments over the merits of various testing approaches and consulting practices.
Many conversion consultants brush aside multivariate testing as akin to simply mixing up the features of a landing page in a blender and seeing what combination works best. They insist that multivariate testing requires little thought and is unlikely to produce good results. They stress their own brilliant breakthrough ideas as being at the heart of any significant increase in conversion.
In reality the matter is much more nuanced.
Thinking vs. Testing
Not everything should be tested. There are many obvious elements (or at least commonly understood things in the conversion improvement community) that should immediately be changed about a landing page. Most know it is common sense, but have not bothered to do anything about it.
The most common reasons for ignoring known landing page problems are that you are too busy, embarrassed about creating the problem in the first place, or too familiar with the page to see it clearly. Once you can get past those issues and are ready to roll up your sleeves, there are several avenues you can pursue to dramatically improve critical landing pages:
- Best practices reviews - Often, using well-known best practices can quickly eliminate the most glaring problems. SiteTuners offers an inexpensive live interactive Express Review of your site via an online meeting. The recording can be shared and circulated to others on your team. The recommendations can be put into practice immediately and will often result in a double-digit percentage improvement in conversion rates.
- Using tools to find problems - Sometimes you might not even be aware of the problems with your landing page. They come in many variations including usability, copywriting and messaging, graphic design, value proposition, trust and credibility. There is an explosion of new tools available to help diagnose problems. They include Click- Tale.com, CrazyEgg.com, CrossBrowserTesting.com, and our own AttentionWizard.com visual attention prediction tool. Most of these tools are free or very inexpensive.
- Using people to find problems - Many times, the only reason you can't find problems is that you are too close to them. You may have even had a hand in their creation. Since you are very familiar with the page, you cannot see it from the perspective of a new or casual visitor. Conducting some informal user testing will provide a wealth of knowledge about what is wrong. Simply sit three to five people in front of your page and ask them to speak out loud while trying to accomplish a pre-defined task. You can even work with a hand-drawn wireframe or post-it note prototype. Tools like Loop11.com can be used to recruit online participants and design tests. UserTesting.com will quickly and inexpensively recruit people and record their sessions for your review.
Multivariate Testing vs. A-B Split Testing
Battles fought over which is the best landing page testing approach can be heated and passionate. Some people claim that the ease of split testing makes it much more likely to be frequently used and, therefore, more valuable. Others swear by the advantages of large-scale multivariate testing.
Split tests have several advantages:
- Ease of design - Simply decide how many versions you want to test, and then split the available traffic evenly among them.
- Ease of implementation - Many content management and software packages support simple split tests.
- Ease of explanation - No complicated analyses or charts are needed to present results to others.
- Useful in low-traffic situations - If your landing page only has a few conversions per day, you simply cannot use more advanced tuning methods.
Multivariate testing has its own advantages:
- Ability to try large number of unique pages - Some systems allow you to test millions of page versions if you have enough traffic.
- Speed of data collection - Running a multivariate test is much faster than running a comparable number of split tests back-to-back.
- Taking context into account - Some multivariate tools take context (variable interactions) into account and consider how well combinations of your page elements play together.
Take a holistic approach and use a bit of brilliance combined with occasional blending to get the best results out of your landing page optimization program. To ignore any of the available tools or approaches would be like playing golf with only a single club in your bag - and just as unlikely to produce a good outcome. So get over the arguments and the posturing and get to work!
About the Author: Tim Ash is CEO of SiteTuners.com, a landing page optimization firm that offers conversion consulting, full-service guaranteed-improvement tests, and tools to improve conversion. SiteTuners' interactive Express Reviews of a landing page can quickly identify major conversion issues. Ash is a frequent speaker at Internet marketing conferences. He is a contributing columnist to several industry publications and websites, and is also the author of the bestselling book "Landing Page Optimization."
