It’s a Sunday afternoon and I’m packing my suitcase in preparation for my trip back to Chiang Mai for the Diggity Marketing annual SEO conference.

My phone starts ringing.

It’s Robert.

Me: Rob, what’s up.
Robert: We did it man! Check your skype.

Me: Wait, you hit it?
Robert: Damn rights.

3 years ago, Robert set a goal, and a pretty audacious one at that.

Goal: $100,000 a month in affiliate income.

Reward:

This is the actual photo Robert sent me on Skype: A 2015 Mercedes c400. His dream car he was able to buy, in cash.

The Proof (from just two of his accounts):

This case study is for 1 of Robert’s websites in his portfolio. 100k was for his total income across all his sources.

We were able to achieve these numbers by constantly running split tests. Over the 3 month period, we ran over 50 tests which allowed us to get incremental compounding gains over time. This process was made a lot easier by having a huge amount of traffic to work with.

The Truth about CRO

There are a lot of misconceptions around CRO and how split testing works. In the CRO Academy, I regularly post examples of wins and losses. One thing that I mention often is that in order to be successful at split testing, we have to disconnect from the urge to always be right. This is confirmation bias and it is dangerous.

This can be hard. Especially if there are no conversion rate optimization experts to give you inputs and ideas.

We are socially conditioned to always want to be right.

Don’t get me wrong, there are conversion rate optimization best practices that are a must on any site.

A few of them, in no particular order:

• High quality images of the product being promoted.
• Clear call to actions.
• Features and benefits displayed to the user to allow them to make an educated decision.
• Making sure all elements look good and display properly in mobile. (yes, alarmingly many sites still don’t do this, in fact over 50% of the websites we audit.)

I won’t get into too much detail about that in this article. I have done that, here, and here.

Once you have what I call a CRO baseline in order, we need to work on getting a large targeted list of experiments ready.

When-ever I ask a potential client if they have ever done CRO and they say no, I get very excited – I’ll explain why.

Just this month, I ran a test using the two variants below:

A simple re-alignment of the calls to action from center to left increased clicks to the calls to action by +23.5%. Seems too easy right? Seems a bit like B.S right? Well, you can grab your phone and do we can do coaching call with CRO experts and see if this is true or not.

This is where it ties back into my story about Robert.

Now, let’s say Robert has finished baseline CRO after reading one of my posts. Clicks to his affiliate account jump 16%. Awesome.

As an example, and to keep the calculations simple. Let’s say Robert’s site was generating $20,000 per month before he did baseline CRO roundup. With a 16% increase that would mean that he would make $23,200 next month.

Same rankings, same traffic.

That would mean that he could take an extra trip somewhere next month, or he could get that new iMac next month that he wanted. No extra costs, just more profit.

Ok now it’s time to set up a split test, he has a goal to get to $30,000 a month from this site by the end of the year and god damn it, he will do it.

That means, he has to increase revenue by another 30% roughly.

Let’s say he tries the same layout as my example above.

He sets the split test up, this is his first one.

He sets his goals as a click on the calls to action button, makes sure the split testing software is installed and functioning correctly. 

Robert starts the test and runs it for 1 hour and because he has roughly 2500 visits a day, he checks again in 60 mins to see if everything is registering correctly.

He sees the initial data after an hour and a euphoric feeling washes over him.

Relative Improvement against original;

Because the algorithm inside VWO doesn’t have much data, it is showing the possible result of the test based on current data could be between -41.1 and +641.9%.

His mind only registers one thing: 600% increase!

That would mean that his possible increase in revenue could push him above $100,000 per month.

Robert can’t sleep that night. His mind is going crazy with ideas on what he could spend all this cash money on, he goes to sleep with a big smile on his face.

How does this Tie into Split Testing and CRO?

Robert wakes up at 6am, and he opens his laptop instantly to check the numbers:

It’s not looking good.

Robert’s mood is instantly crashed, quite the opposite of last night.

The split test is now showing a likely gain of –10 to +32%

Not 600%+ like it looked like last night.

More data has come in and it has evened out the gap between the two variations with the actual test concluding at around 30%+ increases in CTR%.

This is where statistical significance comes in.

You can read about that in more detail here.

Due to us being conditioned to always wanting to be right. When the data tells us otherwise, it creates an uneasy emotional response.

Getting into the right Psychological Frame.

We have to just rely on the data for split testing.

Us wanting to always be right doesn’t have a place here and we need to recognize this early to have any success at this. We have to take a purely rational viewpoint when it comes to split testing. This is where our CRO consulting services can you help you, a lot.

For rationals this may seem quite easy, for the more emotionally reactive, this may take some practice.

When we take on a new client, we don’t have any emotional connection to the website, we didn’t build it, we don’t have any history attached to it.

This allows us to be in the best position to be able to purely rely on data to make decisions.

This is how we view split testing, objectively.

The power of compounding conversion rate increases.

Let’s use our original baseline of 100 clicks for an easy number:

In split test 1, we increased click rates to our calls to action by 23.6%.

New Control Baseline: 124 clicks roughly.

In split test 2, we increased click rates to our calls to action by 6.5%.

That now takes our baseline click rate to 132 clicks.

We are now up 32.6% on our original click-through rate. From just 2 split tests.

This is why we have to disconnect emotionally from the outcome and keep running the campaigns systematically.

Our Emotions Around Money

We have these two emotions that are connected to money:

Greed and Fear.

These two emotions make us perceive data differently at different points in time without us realising it.

The hardest thing about running a split test, especially if you have a larger income website, is managing your emotions throughout a campaign.

We have to set ourself up to expect nothing.

When you run a test, you should not want to expect anything.

When setting up a split test, it is merely a test of Variation A vs Variation B.

That is it.

We have to condition yourself to not have to be right in our presumptions of what will perform better.

Organizing Your Workflow.

In order to be successful at testing, especially at scale and over the long term you need to follow a very clear process:

Like any good business, you need a systematic process that can be repeated over and over again, without those pesky emotions coming into play.

You need to know;

1) Which tools to use, and

2) How to use them to get results.

Understanding the process.

When I get asked to look at someone’s website and tell them how to improve conversions, my simple answer is always:

I don’t know yet.

I need to implement my process that works out what works.

The example above of increasing clicks by 20% just by moving a button to the left, may not work at all on another website. Does this mean we always start from scratch with every website and have no previous takeaways? No.

Because what we do have is a long list of layouts, buttons, and combinations that have worked well in the past on similar sites for us to test.

As time goes on this list grows and it refines our process, so that we get it right the first time, more and more often, but we always test.

It’s all a numbers game.

I’m going to say it right now so it’s clear:

The more traffic you have on a site, the more success you will have with split testing.

Let me explain why:

Let’s call the first site Site A.

Monthly Revenue: $1000

Monthly Visitors: 5000

Highest Traffic Page: 2300

Normally at CROguy we focus on the top 5 pages of a client’s website, but in this example we’ll just test one page.

The top 5 pages, in most cases make up 80% of the revenue of the site. Not all the time, but that is what the data has shown us over time.

Ok so let’s set up our first test on Site A.

Control: CTA Button: View Price

Variation 1: Check Price

In order for us to declare a winner for this site, we would need a few things.

1. Enough difference in clicks on one variation over the other.

2. Enough views for a statistically significant winner. (So we can be sure the results were not due to chance.)

Let’s say it takes 2 weeks to declare a winner in this split test.

That means, during the split test, the control had 575 visitors and Variation 1 had 575 visitors.

Because you have waited 2 weeks to get this result, you would damn well want it to be a winner, right?

Wrong, Variation 1 was –10%

You wouldn’t be in the best mood if you just spent two weeks waiting for this test to conclude and it was a loser.

F!@k split testing.

Looks like emotion just came into play didn’t it?

Now let’s rewind a bit.

It’s far easier to get better results with sites that have larger revenue and more visitors.

We can run multiple tests per week, get quicker feedback and can find those winners quick and faster.

That is how we got such big increases for Robert:

What we did have was a plan, documentation of results and management of each campaign. We also took advantage of the available conversion rate optimization software.

We didn’t test just one page on Robert’s site, I believe it was closer to 50 tests. If you didn’t have a process to follow, on that scale, it would be impossible to manage.

Due to Robert having over 200,000+ unique views a month, it allowed us to very quickly run a lot of tests back to back and uncover the winning variations thats allows us to increase his revenue for this site by over 100% which in turn, helped him to reach his 3 year goal of $100,000USD per month.

Tools we use:

• Hot jar
• VWO
• Google Calendar
• Slack

In conclusion.

Managing your emotions can be easy once you have the right systems in place.

Have a process allows you to make quick, relatable decisions based on data.

The data holds all the secrets. Don’t let those pesky emotions get involved.

Click here to join the CRO ACADEMY where we ask, help, share and collaborate on all things Conversion Rate Optimization.

Until next time,

Kurt Philip

CROGuy