Five Steps to Building an Optimization Program Leadership Can't Ignore

Personalization is an integral part of a successful digital marketing strategy, and optimization plays a big role in making personalization happen at scale. And yet one of the main issues I hear over and over from practitioners is that optimization is a secondary player in the marketing organization.

For example, the optimization program manager at a large retail company faced pressure from senior leadership to run tests that she knew would barely move the needle. To make matters worse, the important tests she had planned were canceled due to low bandwidth and design resources. She knew that the company's optimization program had high business potential, but she wasn't in a position to influence how it was run or how the organization used it.

This might feel all too familiar for many of you. So, I consulted with my colleague, Debra Adams, who consults on many optimization programs for customers at Adobe, and here are the top five steps we came up with to help every marketer transform their optimization program from a secondary function into something leadership can't ignore. 

Step 1: Build a strong case for testing

In a perfect world, everyone in the marketing organization would instantly understand the value of testing features before they're released, but more often than not, you need to make a compelling argument that your testing as the potential for big impact and return on investment (ROI).

Consider using the following to build a strong testing business case:

• Leverage real business examples of the impact and ROI from testing.  
• Highlight calculations of possible ROI. Your numbers should be based on your traffic levels as well as the average revenue per visitor (RPV), average order value (AOV), and conversion lift typical of your industry. You can also compare possible ROI from optimization against known ROI from other digital investments like display ads and search engine marketing.
• Include supporting research from analysts and benchmark data from self-assessment tools.

Step 2: Overcome the data predicament

In today's business world, we're all about data, and we often use it to justify why features or products are important. Unfortunately, not all data sources offer trustworthy or high-quality data, so many people are understandably dubious of data-backed claims.

To get buy-in for optimization, demonstrate that your business case and test data are credible by:

• Identifying a spokesperson who is experienced with data, like a data analyst, data scientist or statistician, but also has the ability to speak the same language as the business executives.
• Ensuring your data tool is trustworthy and set up properly.
• Tracking the right metrics by making sure they clearly tie to the business bottom line.
• Use the same metrics across tests to allow apples-to-apples comparisons.
• Use unified profile data that ideally spans your offline and online channels for a holistic view of your data.  
• Make sure the test results are clear, pertinent and valuable to the business.

Step 3: Make sure your tests are valid

When someone pokes a hole in your test design or results, the organization may lose faith in that test-perhaps even in your entire program. To produce valid results, you need to follow the established rules of statistical test design including:  

• Identifying the dependent and independent variables in your test.
• Limiting the number of elements within test variations. If you have too many, you can't isolate which element elicited a visitor response.
• Running the test long enough for results to be statistically valid. Use a sample size calculator to determine the amount of time needed to reach an ideal traffic level.
• Limiting the number of test experiences for your site traffic levels.
• Ensuring the test experience differences are bold enough to elicit a measurable user response.
• Tracking the metrics that indicate true business success.

Step 4: Get your seat at the table before big decisions are made

Your sway over big business decisions is severely limited if testing comes into the decision making process after priorities are set. As an optimization lead, you need a seat at the table as these discussions and decisions about what tests to run and priorities are being made-not after.

This forum gives you the opportunity to explain why certain tests and metrics aren't useful, share the testing roadmap and strategy so priorities can be set, and demonstrate the business value of using data, not marketer's intuition, to determine and deliver the experience customers want.

Step 5: Reinforce the value of your testing program

Once you've earned your seat at the table, you need to constantly reinforce how your program contributes to business success. You can do this in a number of ways:

• Create a year-end summary that outlines all the tests run, how many moved the needle, and how many didn't. Describe how the program measured up against its goals.
• Regularly communicate to the organization what your program is doing. Try gamifying optimization - letting employees guess which test experience won and giving prizes for guessing correctly.
• Show designers the impact of their creative to inspire them and help them learn what types of creative work.
• Engage stakeholders in submitting test ideas and give them credit and recognition when ideas produce big wins.
• Plug into marketing campaign teams. Get in front of new campaigns. As a side effect you may spur greater adoption of and involvement in testing.

Recognize when you've succeeded

You'll know you've transformed your optimization program from a sideshow to part of the main event when your company believes this about testing:

• It's valuable and necessary
• It should have dedicated resources
• Development schedules should accommodate it

Getting to the point where optimization is simply part of the company culture takes work and determination. It's well worth the effort, though, when you repeatedly demonstrate how your testing prevents the business from implementing poor features and changes and helps them deliver the customer experiences that positively impact the business bottom line.

About the Author
Drew Burns is the principal product marketing manager for Adobe Target, part of Adobe Marketing Cloud, and works to evangelize the practice of iterative testing and content targeting in the digital marketing world.