Unlocking Your Company’s Growth Engine w/ Conversion Rate Optimization

If you’ve heard of Dropbox, Eventbrite, LogMeIn or Qualaroo, you’re familiar with my work.

I’m Sean Ellis & I’d like to tell you how Conversion Rate Optimization played a critical role as the growth engine for the companies I’ve worked with. I’m going to lay out the specific tests & processes I’ve used to grow these startups into incredibly valuable businesses.

How important is CRO? It was the key to our success in scaling LogMeIn to a publicly-traded company, and helped us optimize our referral program at Dropbox, whose latest funding figures peg it’s value at a rumored $10 billion.

The Difference Between Testing and Unlocking Growth

Since you’re reading this on ConversionXL, I’m going to assume that you are already testing regularly, that you implement A/B or multivariate testing regularly, and that you’re tracking the performance of your landing pages and funnels and optimizing them to one degree or another.

This is great, and you’re leaps and bounds ahead of most marketers.

But I’m willing to bet that you haven’t yet gone far enough to turn it into a sustainable engine of growth for your business. Many people get hung up at the testing stage.

Running A/B tests, they assume that they’re “doing CRO”, and to some extent they are, but the lack of a systematic approach to continual improvement leaves them with shallow victories that fail to translate into long-term growth.

Why is this? They lack a system.

Without a system, CRO might result in short-term, campaign-level wins but it very rarely translates into a sustainable growth engine.

“Conversion is a process not an event” – Bryan Eisenberg

The Power of a System

If you follow American football with any regularity you’ve surely heard of Joe Montana & Dan Marino.

image source

Both are Hall of Fame Quarterbacks that played in the 1980s & 90s. Their stats are remarkably similar in many ways. For most, Dan Marino holds the edge, except for one… SuperBowls.

Joe Montana won four titles in his career. Marino? Zero. Why? It’s all about the system.

San Francisco 49ers Coach Bill Walsh said of the two quarterbacks (of which he coached Montana)

“Joe Montana was a product of the system. Dan Marino was the system.”

What he meant by this is that Montana flourished inside of a system designed to produce championships, Marino wasn’t a part of a system or a process. Marino was great, but he didn’t have a framework around him to make him successful.

The West Coast Offense

Bill Walsh and the 49ers ran a style of offense called “The West Coast Offense.”

The offense, managed by Montana, was designed unlike other professional football offenses at the time. It favored short, precise passing routes instead of running the football and throwing low percentage passes deep down the field.

image source

These short passes led to a much greater efficiency for the quarterbacks, including Montana, who ran them. They traded big plays for a series of high probability completions that compounded on one another quickly as they moved down the field to score.

There were more unique elements to this style of play, including calling for the quarterback to reduce the number of steps taken before releasing the ball. Called the “drop”, the West Coast Offense called for three steps instead of the traditional five steps.

This shorter drop dramatically sped up how plays unfolded and gave the 49ers and Montana many more plays over the course of the game.

In addition, the playbook for the offense is fairly simple. It is based on a handful of passing and running plays that have variations to them based on what the defense is playing against the offense. The West Coast Offense tests quickly and then adjusts to results on the field to find new opportunities to exploit the defense.

“The West Coast Offense” was essentially Conversion Rate Optimization for American Football.

A focus on speed, deploying tests quickly; a focus on learning, starting with simple, clear goals and then taking feedback and making adjustments in the course of the game; high efficiency, each play compounds on the previous, and the ability to run more plays in a given game gives the 49ers a massive advantage over competitors who have fewer chances.

There was perhaps no greater quarterback than Montana at running this style of offense. He was a deadly accurate passer, which made him the perfect fit for the offense, and his ability to run this fast paced attack with precision is what made him great. Marino on the other hand, was great within the traditional system, but it still never won him a Superbowl.

Efficiency, precision, tempo, learning, speed—the elements of a great quarterback are the same elements of a great CRO program.

Be a Montana, Not a Marino.

If all you’re focusing on is the outcome of single campaigns at a time, your current testing and optimization is like Marino.

You may be racking up wins on landing pages and email campaigns, click through rates and quality scores; but if you’re not adding long-term, quarter over quarter gains to the business, you’re missing out on the sustainable, compounding growth potential of CRO.

Systematic Optimization of the Core User Experience is the CRO Growth Engine

The optimizations that really move the needle are the ones that have the greatest impact on getting more of the right people to the must-have experience of your product. This means the elimination of random testing on elements that don’t matter.

Making CRO a growth driver requires a holistic view of optimization. When you go beyond conversion optimization of marketing campaigns, to full funnel improvement, you move from testing to true growth driving activity.

To do that, you need to test throughout the funnel and use the wins to create compounding advantages that lead to long-term success.

Conversion Optimization Systems

The good news is, there is no shortage of frameworks for conversion optimization. Here are three approaches to CRO that you can use to go from one-off testing to a consistent driver of growth.

PIE by WiderFunnel

The PIE framework is an optimization approach that helps you identify what to test next through a straightforward prioritization method.

PIE stands for:

P — Potential

• What’s the potential lift if this test succeeds?
• What’s the upside and value in running the test?
• Does it have the chance for big improvements or incremental improvements?

Knowing ahead of time the potential for success (given the data) helps you to get yourself & others excited not just about testing, but also understanding the data, and coming up with educated hypothesis based on what they see.

I — Importance

• How important is this page or funnel that I’m testing on?
• Is it going to impact a large portion of my traffic or just a small section of my website?
• Am I testing the core features and user experience or am I dabbling at the edges?

Answering these questions will help you further understand the potential, but also evaluate the risk associated with the test.

If it’s an extremely important funnel, for example, you might decide to run your test with a smaller sample size (20%) and over a longer period of time to help others feel comfortable testing something so critical. Conversely, if the funnel isn’t as important, but the potential for lift is high, you might go into the test guns blazing for bigger wins.

E — Ease

• How hard is this test to implement?
• Can I run this test myself through a testing software like Optimizely?
• Or does it require weeks of development work to roll out the test?

With this approach, you rank each test idea on a scale of one to ten across each element with ten being the most valuable, most important or easiest. The tests with the highest average value are the ones that you should do first.

As you roll through each of these tests, you systematically move on and work on the subsequent highest prioritized test. By continually ideating on new test ideas and re-prioritizing them, you end up with a continual process of always running the most important tests.

WiderFunnel uses the PIE framework to drive big gains for their clients, such as Electronic Arts. Tasked with improving the registration process for an online golf game, WiderFunnel identified 13 potential variables that were impacting conversions and sign ups.

Before:

After:

Through this prioritization process, WiderFunnel created four new sign up tests designed to eliminate anxiety and distraction to improve conversion.

The result was a 12.8% lift in sign ups by simplifying the sign up process and designing for the ‘get to the gameplay’ mentality of the users.

Plan, Measure, Improve

Bryan Eisenberg, the godfather of web optimization recommends the plan, measure, improve approach to testing, along with a prioritization system for figuring out which test to run and when.

It starts with planning, which Bryan recommends beginning by answering three questions:

1. Who are we trying to persuade?
2. What action do we want them to take?
3. What action do they want to take? (not always the same as #2)

Once we’ve answered these questions, then we need to define the test and how we’ll measure it.

This is based in another three questions:

• What action do we want them to take and how will we measure it?
• What page will we test?
• Where/how do we judge success?

Finally, in the measurement phase, we need to look at the results of the test and use that to feed back into the framework to help us identify what to test next.

In addition to this loop, Eisenberg recommends prioritizing tests by these three factors:

Time — How long will it take to get statistically significant and conclusive results from your tests. How fast can you get learning, call a winner, and move on?

Impact — What is the revenue potential or cost savings for these tests? Are the gains likely to be incremental or do they have a chance at being game changers?

Resources — What will it take organizationally to get these tests done? How many people hours, development time, testing time, etc. will it take to launch these tests.

Using Eisenberg’s formula, you rate each on a scale of one to five, with five being the test with the greatest velocity, impact and lowest cost in terms of resource. Each score is then multiplied by five, and the resulting total is used to identify the top performing tests.

Using this prioritization strategy, Dell Computers, a client of Eisenberg, found a very simple but meaningful win in its eCommerce check out flow. In the product “configurator” Dell, on the recommendation of Eisenberg changed the label from ‘Learn More’ to ‘Help Me Choose’.

This simple test resulted in millions of incremental revenue for Dell as more people understood and used the configurator to build and buy their computers online.

If we look at this test in the context of deploying conversion optimization at an enterprise company such as Dell it makes a lot of sense:

Time—on a high traffic site like dell.com in the core user experience, it wouldn’t take long to get statistically significant learning about this change. It could be tested and finalized quickly, win or lose.

Impact—the product ‘configurator’ was the key mechanism to Dell’s eCommerce success. Running a test right in the most important part of the shopping experience gave it a high likelihood of having an impact.

Resources—this is the most important consideration in a large corporation like Dell. They didn’t start with a test to redesign the entire funnel. They made a small change that was easy to get approved and moved into production before going on to bigger gains.

Eisenberg’s incorporation of velocity is a critical to turning CRO into growth. Organizations that can run more tests build up incremental gains over time that result in out-sized edges over longer time horizons. Be sure to include some accounting for testing velocity in any framework you adopt.

Analytics, Insights, Optimization

The framework I recommend is one that includes analytics, insights and optimization that goes through a never-ending loop of continual learning and optimization. By using analytics, we can understand which parts of our funnel have the most friction and drop off—and therefore the most opportunity for improvement.

I like to start the prioritization process by looking at the core user funnel that leads them to the must-have experience of the product. Within that funnel I look for opportunity. Eisenberg is instructive here.

He recommends looking at the following:

• Your top 5 high bounce rate pages
• Your top 5 high exit rate pages
• Your top 5 lowest time spent pages
• Your top 5 key pages (i.e. checkout, cart, registration, top product)

Analytics can only tell you what’s happening, but it can’t tell you why. Which is why I include Insights in my CRO framework. I’ve always been amazed at how little marketers actually talk to their visitors and customers.

Throughout the core steps of the user experience, I ask visitors what stops and persuades them.

Of course, I do it with Qualaroo, but there are other tools out there you can use to do the same thing. You want to ask both successful and unsuccessful people.

You can collect feedback with exit surveys, targeting visitors on site based on inbound traffic source or previous page visited logic, or you can do it via live chat, or post-purchase via email surveys, the opportunities to gather feedback are many.

What you want to find out is what are the deal breakers or confusion points that lead to abandonment? What are the real selling points that keep the user experience moving right through ultimate conversion.

The key to getting solid insights is to ask great questions.

To people who visit your high bounce rate pages ask:

• What were you looking for when you visited this page?
• What did you search for to get to this page?
• What are you trying to accomplish by visiting this page?
• Does this page make you want to sign up for our service?

These questions can be asked via an exit survey that triggers when they go to leave the page, or after they scroll half-way down the page. The key is to try to gain insights about the visitor mindset and to uncover the reason behind the intent mismatch that is leading to your poorly performing page.

To people who successfully complete your ultimate conversion:

Question 1 – What Almost Stopped You From Completing [The Conversion]?

This question will tell you where your successful visitors almost lost faith, got stuck, or gave up. While they made it through, you are guaranteed to find many more who never make it over that hurdle. You’ve just uncovered a key optimization opportunity that will yield ongoing results day in and day out.

Question 2 – What Persuaded You To Complete [The Conversion]?

This question will tell you what resonated the most with your visitors. What convinced them to forge ahead and take the plunge. You can use this information to optimize your messaging, site copy and appeals on the site.

Imagine you’re a SaaS provider of project management software. On your marketing site you tout the number of projects that you allow users to have. You might find that your customers are most persuaded by some other feature or element (say the number of users that can participate). You use this feedback to improve messaging, attracting more customers with a value proposition that is more relevant to their needs.

This type of insight is used in conjunction with the analytics data to inform the best potential next test to run. With the context of user feedback you can now run smarter tests that lead to bigger optimization wins faster—improving the velocity of organizational momentum and compounding interest of incremental wins.

Sidenote: This can be done with a tool like Qualaroo, an embedded google form (free) or as a part of your email onboarding series if the conversion involves capturing an email address.

Developing an Organizational Habit

It’s no coincidence that each of these frameworks works in a loop.

As Tim Ash says:

“There’s no final answer in CRO. You’re never done with a test, you’re just changing the mix of what you’re testing.”

Only when you get to this stage of organizational understanding does CRO become a sustainable long-term competitive advantage. Eisenberg calls it your ‘Organizational Metabolism’ which is roughly defined as the number of tests you can get through in a period of time.

This testing habit or culture is what creates the growth engine. Organizations who run four tests a day are learning eighty times as much as the average organization running five a month.

Compound that over a year and they run 1,040 tests in a year to their competitor’s sixty. Is a massive advantage.

So there you have it.

To go from testing pro to CRO-driven growth you need a system and you need the corporate agility to move through the system as fast as possible. Speed, the CRO habit, the metabolism to run tests regularly, successfully and with high velocity; this is what creates the growth engine for your business.

Getting Started With CRO Driven Growth

But now the question is, how do you get started?

Well, that all depends on where your organization is at when it comes to conversion optimization.

Tim Ash suggests that each organization is at a different stage of maturity when it comes to conversion optimization.

Some are toddlers learning to crawl, others are awkwardly learning to walk & others are sprinting like Usain Bolt.

It’s up to you to figure out where your organization is on its journey in order to determine what steps are necessary to turn your current testing efforts into long-term growth drivers.

If you’re at a company where CRO is restricted to marketing campaigns its up to you to be the champion for expanding CRO beyond advertising to the rest of the organization.

How?

Start With Small Wins.

Use the results of your efforts as the rationale behind running deeper and more integrated tests.

Use landing page wins as the rationale for testing the home page or product pages.

Use one of the prioritization frameworks above to find the potential big impact opportunities that can be done for the least amount of investment possible.

Get small tests scheduled and share the results widely, building buy in along the way.

If you’re already running successful tests, share those with management along with an appeal to run more meaningful tests throughout the funnel. A proposal for a test that is backed by a track record and a well-reasoned hypothesis and potential benefit prediction can go a long way in making things happen.

Oli Gardner shared how the simple tweak of removing links in a bulleted list on a landing page improved conversions 42.3%.

It’s these small tests that lead to big wins that will be the driving force that turns CRO from a marketing practice into an organizational one.

Double Down and Compound Gains

My most successful long-term optimization wins have come from doubling down on what is working with further optimization, not on fixing ‘broken’ or inefficient channels.

At Dropbox we knew that paid acquisition wasn’t working; but our high customer satisfaction and organic word of mouth was a natural driver.

Instead of focusing on how to make ‘paid’ work, we spent all of our efforts optimizing and amplifying the natural acquisition channels that were working.

By doubling down on what was working we saw outsized gains from our optimization efforts. As Peter Thiel said:

“Just as it’s a mistake to think that you’ll have multiple equal revenue streams, you probably won’t have a bunch of equally good distribution strategies. …It is very likely that one channel is optimal.

Most businesses actually get zero distribution channels to work. Poor distribution—not product—is the number one cause of failure.

If you can get even a single distribution channel to work, you have great business. If you try for several but don’t nail one, you’re finished. So it’s worth thinking really hard about finding the single best distribution channel.”

Doubling down is the key to making CRO a growth driver.

Much like money, optimization wins compound over time. And just like compound interest creates more return on more capital, CRO creates greater advantages the more optimized the funnel, channel or page becomes.

Focus on the Must-Have Experience

CRO is not about pushing failure down the funnel. Just because you optimize one step doesn’t mean you’re impacting growth.

Getting the wrong person one step closer to a goal they’ll never reach doesn’t translate into gains for your business.

Only when your CRO efforts are aligned with getting the right people to your must-have experience will your efforts turn into a growth engine. You must balance your optimization of any particular part of your funnel with the overall goals of your business.

Once you are able to look at optimization results in the greater context of the growth of your company, you’ll truly have the ability to impact growth through your CRO program.

Unlocking Growth with CRO

CRO is having it’s moment, no doubt, but before the term becomes completely misunderstood & bastardized by a portion of the industry who moves from buzzphrase to buzzphrase thinking that is what builds an online business…

::ahem:: growth hacking ::ahem::

…let’s look into the future & realize that the true potential for conversion optimization doesn’t lie in one off tests, but has the ability to work its way into every level of the organization to make for a completely unforgettable customer experience.

Now comes the hard part: turning CRO from a campaign-level activity into an actual engine that drives long-term benefits for their company.

Let’s turn the wins and optimizations we’ve found on the marketing side into long-term advantages for our business that unlock sustainable growth and success.

You ready? Let’s do it.