4 CRO Split-Testing Platforms You Need to Know About for 2019

Posted by Kurt Philip In: Blog, CRO Tips, Heatmaps, Resources 6 Comments

Every week, I get asked: “What is the best CRO split testing software”.

The answer purely depends on your specific goals and requirements.

All the split testing platforms below are essentially different tools to achieve the same goal. To decide software is best for you, consider a few things:

Are you doing CRO on one or few of your own sites or are you running an agency?

Will you be running CRO campaigns all year round or just every now and then?

Do you have large profits to re-invest into CRO or are you boot-strapping?

Worry not Mr SEO, there is a ‘right’ CRO software for you.

All the strategies we have shared here at Convertica can be tested and implemented with any of the 4 split testing service providers below. The only thing that will put one over the other is your specific needs. I will work through each of our recommended solutions outlining the positives and negatives of each so you can come up with your own conclusion to see if it is suitable for you and your team or not.

On top of that, I will work through each solution to see how easy it is, step by step to set up thevery  same A/B test.

Please note: There are no affiliate links in this page. This is purely to share our experience with you guys on the different split testing software out there so you can decide which is the best to suit you. 

Fresh Marketer

Summary:

Freshmarketer are a relatively new player to the split testing market. Previously known as Zarget, they provide a simple and easy interface for people looking for a no-nonsense split testing tool. There are also a bunch of different features.

Once we log in to the backend, we can see a menu of different features available (albeit some limited on the free trial of course).

Our favorite features are here, including;

  • Heatmaps
  • Funnel analysis
  • Form Tracking
  • AB Testing
  • Split URL testing

So, just like we did on the other split testing software, we will go through and set up an A/B test, and see how simple it is. 

Here is how we set up a simple A/B split-test:

The nice minimalist background makes it very simple to navigate around.

Let’s click the AB testing icon on the left to get started.

Next, let’s create an A/B test.

Next, I get alerted to this;

I was initially in Firefox so let’s continue the process in Chrome.

After I log in, I get prompted to install their plugin, so let’s do that.

Now that that’s, done, let’s continue on with the setup.

So far, pretty straight forward.

Next, I am brought straight to the visual editor.

What is different to the others here is that it also asked me my test name and hypothesis within the editor, which I kind of like.

Next, we hover over the text area I want to edit and left click on the text, then select these options below.

Now that’s done, let’s continue on to set up goals.

So far, really simple, intuitive and straight-forward.

Next, the goals page.

We will be setting it up just like the rest of the tests. Tracking a success URL that contains “success” in the URL.

We will also target all visitors for this test. I like how this feature is added here and the way it is laid out, great for beginners.

On the next page, all the settings are laid out clearly.

Let’s go ahead and launch the experiment.

I am then prompted to verify the code integration, which we haven’t done yet.

Once, you have added that code to just after the <header> you will be good to start your test.

Rating: 8/10

Pricing: $

Suitable for: Beginners

Positives:

  • Super simple set up.
  • Intuitive backend.
  • Tonnes of segmentation options

Negatives:

  • No API connectivity.
  • No RAW data export so you are limited to their reporting backend.
  • Basic Reporting (can be a good thing for beginners).

 Overview:

Freshmarketer is an excellent option for people who own 1-5 websites and are new to split testing. Super intuitive set up process and minimal backend allows you to explore CRO without getting confused by the process. 

Get started with Fresh Marketer here.

Optimizely

Summary:

I actually started my CRO & Split testing journey with Optimizely. Saying that, I haven’t actually logged in to their backend since moving to VWO. So, I am going to go through with a completely new account and run through setting up the same AB test as I have on all the other providers.

It looks like they have made a few changes to their homepage, so I am excited to see what the backend looks like now.

So I go through and set up an account, and then I get this.

Well, that was disappointing. It’s just a lead capture.

You actually can’t create an account without someone ringing you first.

Similar to VWO.

They must just be focusing on enterprise clients.

I will update this post once I get access to the backend.

Here is how we set up a simple A/B split test: TBA

Rating: TBA

Suitable for: TBA

Positives: TBA

Negatives: TBA

Overview: TBA

Get started with Optimizely here.

Google Optimize

Summary: 

The coolest part about Google Optimize is that because it is a part of the Google Suite of Companies, you can run it under your Google account. On top of that, you are able to integrate it seamlessly with Google Analytics which, for some people (especially eCommerce website owners) is a huge bonus.

What I really dislike about Google Optimize, and which is very surprising (considering it’s Google owned) is the UI and unintuitiveness of this platform. IMHO, there is a lot to be improved here.

Here is how we set up a simple AB split test:

Once you have logged in to the Google Optimize home-page you will arrive at this screen.

Click create experience.

Next, you will be asked to enter some information about your campaign.

Then click create.

Then, click create variant.

Then click where it says Variant 1 and it will ask you to install the Chrome Extension, follow the steps for this and then you will have to reload your campaign.

Again, pretty frustrating user-interface here. It took me a while to work out what I had to do after each step.

You will then be loaded into the visual editor.

Having a programming background, I do actually like this visual editor. The fact that it overlays all the container identifiers into the visual editor is handy.

We jump in and simply click the section we want to test, which again is the headline above our lead capture.

Pretty straight forward.

So was pretty simple to change.

Click done, then top right hit save, then done again.

Now I wanted to go and set up goals. The same as we did for VWO.

This next part is what I found really frustrating from a user experience point of view.

I actually couldn’t work out where to set up goal tracking.

This is where other SEOs might get frustrated. Please make sure you set up your account in the same account as your analytics.  Due to us having many different accounts, you may have to go back through and start from scratch. Which is what I had to do.

OK so I went back through and set up the test again. In the correct account.

Back to setting up goals.

After you have finished setting up your tests, scroll down and link to Analytics.

You then have to add some extra tracking code to your Analytics tracking code so it can track properly.

You then need to set up an objective. Which needs to be a goal already set up in analytics. At this point I am getting quite frustrated at how broken this process is compared to the rest of the software available.

So I jump back into analytics and set up some goals.

Now set the URL that contains /success like the rest.

Now, jump back to Google Optimize to see if this is all set up right.

Nice, looks like its all showing up fine.

And….. that’s about it. Not the most pleasurable experience IMHO.

Rating: 6/10

Pricing: Free for a small amount of test and pricey if you want to run any volume, just like the rest.

Suitable for: Those who like the google analytics integration.

Positives:

  • Integration with other Google products
  • Free for entry level

Negatives:

  • Bad user interface
  • No clear pricing

Overview:

Google optimize may suit some people but it was a very broken process for me to set up an account. While google loyalists may like Optimize it isn’t for me at all.

Get started with Google Optimize here.

VWO

Overview:

This is what we use here at Convertica. It is robust, has great raw data reporting and the support team is great at customizing a solution to suit our company. The downside is: You need to pick up the phone and talk to one of their sales reps so they can build a package that suits your specific needs. Sure you can set up a trial account but this has very, very limited features and is used by the VWO team as a lead generation funnel for them to find the bigger clients that they can upsell their enterprise accounts.

Still, this software is our favorite.

Due to the API connectivity, we were able to build a dashboard that allows us to quickly review each client quickly and easily in one space. Saving us tonnes of time and allowing us to keep an eye on each and every split test we have running at the same time. At Convertica we run hundreds of tests every single day, and the customisation VWO allows is crucial to our effectivity.

See below;

Also, with the enterprise account, we are able to pull raw data and track different goals depending on what is required for each of our clients. We’re able to deliver a 100% tailored CRO approach to each client with this flexibility.

Here is how we set up a simple A/B split test in VWO:

Pretty simple stuff, when you are logged in, click these two sections.

Then;

Then it will load the live editor.

At this point we click the area we want to edit.

We then click the container in which the text is located and click edit HTML in this case.

I will then add in “FREE” to see how this will impact conversions.

Now because we want to track successful submissions of our form, we need to track URL’s that contain “success” in them. We do this as we have different form segmentations depending on what the user selects to allow us to track things better.

That looks like this.

Next we go through and preview each section before starting to make sure they are rendering right.

If all looks good, you can start your campaign.

Make sure you install your smartcode right after the <head> in your theme header. It is important that it is installed on all pages (like the success page) that you are tracking for it to track correctly.

If you don’t have the code installed, it will tell you it can’t find it and show you the code to install.

As easy as that. I love how straightforward VWO is.

Rating: 8/10

Price: $$$$

Suitable for: If you are managing a very large personal portfolio or 20+ clients at a time.

Positives of VWO:

  • Super intuitive set up process.
  • Great custom support
  • Will tailor a package to suit your needs by monthly visits and more.
  • Cost effective when run at scale (20+ clients)
  • Has many other solutions like heatmaps, recordings and funnel flow in one platform.
  • Great API and raw data functionality.
  • Can control tests within slack with the proper API configuration.

Negatives of VWO: 

  • Price inhibiting for people managing under 5 websites.
  • You need to get on a call if you want to unlock most features and go through the VWO sales process which can be frustrating for small portfolio owners.

Get started with VWO here.

Conclusion

We went through and set up an account from scratch with VWO, Fresh Marketer, Optimizely and Google Optimize. We wanted to look at it from someone starting from scratch.

For us there are 2 clear favorites. Fresh Marketer for those starting out and VWO for those that can spend $1,000 a month or more. It’s been 4 days and I haven’t even had a single email from Optimizely and I just flat out didn’t enjoy the Google Optimize Experience.

Now that you have a few options to look at, go through and set up an account your self and see what suits you. You may be surprised.

If you are serious about growing your wealth in 2019, especially if you are looking to sell in the next 6-12 months.

YOU NEED TO BE SPLIT-TESTING.

2 Likes

Comments: 6

  • Matt Travers
    2 weeks ago
    Reply

    Nice post Kurt, i’ve used all the tools you reviewed except Freshmarketer, i’ll have to give that one a spin. Ive pretty much been using Google Optimize for all my tests and agree the UI could use a major overhaul.

    • Kurt Philip
      Kurt Philip
      1 week ago
      Reply

      It was so painful even just setting up this post with Google Optimize. Even for an experienced CRO.

  • Gareth Daine
    2 weeks ago
    Reply

    Great post, Kurt.

    Fresh Marketer sounds awesome. Especially, for someone like me who’s just getting started with CRO.

    Cheers, dude.

    • Kurt Philip
      Kurt Philip
      1 week ago
      Reply

      Exactly mate. That’s why I featured it. It’s actually a good piece of software for being so new.

  • Adam
    1 week ago
    Reply

    Good round up Kurt. I like the fact you went from fresh accounts, as the sign up process can sometimes be very frustrating when you just want to get stuck in. Ironic considering their service is about CRO 😉

    I have used VWO in the past (few years ago) and worked well. Fresh Marketer is new one and so may check this out.

    Cheers

    • Kurt Philip
      Kurt Philip
      1 week ago
      Reply

      Yeah exactly. Glad you liked the way I did this. I think alot of people get confused by the process (especially with Google Optimize) and then don’t do anything.

Leave a comment