Conversion Optimization Round-up – February 2022

In this month’s Round-up we bring you valuable information regarding A/B testing and how, when, and where to use it.

Today you will learn from Guess the Test about using the top display banner or notVWO looks at the widely successful Bear Mattress, and how they are improving revenue. Marketing Sherpa shares three-speed to value marketing case studies featuring A/B testing. And finally, Medium offers us a wealth of knowledge all about- you guessed it, A/B testing.

The articles we have set forth today will explain why A/B testing is so vital for the success of your business. As Chris Grosser says, “Opportunities don’t happen. You create them.”  So, sit back and get inspired.

With or Without the Top Product Display Banner

A/B testing is what Guess the Test is all about, and this month is no different. The company used for this month’s test was a guitar company named SweetWater. The KPI (Key Performance Indicator) was engagement measured by unique views of guitar product detail pages, Average Order Value (AOV), and purchases of guitars.

Version A used a top-placed display banner above the featured item, and version B did not. Which one do you think came out above? Yup, you are right – version A was the clear winner. The study ran for one month and saw over 2.2 million sessions. The traffic from these visits was split evenly 50/50.

Guess The Test

Compared to the version without the top banner, including it led to:

  • 118% conversion lift in unique views of guitar product detail pages 
  • 5.72% increase in Average Order Value (AOV)
  • 6.64% rise in purchases containing a guitar

Time To Value: 3 Speed-To-Value Marketing Case Studies

We love sharing the case studies from the Marketing Sherpa. Today is all about rapidly getting the attention of your potential customers and providing them the value they require before they help send your bounce rate soaring.

The first case study brings you to Talkspace, an online couple’s therapy that used an A/B test to determine if shortening an end-of-quiz animation would increase conversions. Well, it did, by a staggering 60%. By getting out of the customer’s way, Talkspace brought the information the customers needed faster, in turn generating more returns.

Marketing Sherpa

Version A tested an animation for signup at 8 seconds, version B shortened this process to 4 seconds. Results showed version B outscored version A by 60%. In addition to testing different video lengths, you should test different video locations and run testing on the checkout flow.

How Bear Mattress Leveraged Cross-Sell Opportunities with VWO and Increased Revenue By 16% – Success Story

We love to share success stories with you, especially when we see huge numbers like these. Founded in 2014, this mattress company with a vision to change the way we sleep, aspired to some massive goals. And with continued testing and adjusting, they are getting there.

The goal of their latest test was to aim at increasing revenue by focusing on optimizing the cross-sell flow on the mattress product detail pages of the website.

They determined using VWO Insights that:

  1. There wasn’t much visitor interaction within the frequently bought with mattress section. Which consists of the cross-sell product listings.
  2. Overall clicks on cross-sell products were fewer compared to the other elements of the page.
  3. Images weren’t showing with items present in the ‘frequently bought with mattress’ section of the page, which was impacting the engagement.
  4. The copy was not customer-centric but product-centric. It did not pique interest or give enough reasons to customers to consider buying the product.


Overall, the variation outperformed the control with a 24.18% uplift in successful purchases and a 16.21% increase in revenue. Take a look and see how you can implement a test like this for your website.

A/B Testing for Product Managers

This article is all about A/B testing (aka: controlled experimentation). Andrew Hon from Medium shares with you a talk he gave to product managers from around the world. He explains the vital reasons why A/B testing is necessary for the growth of your company.

Three main points:

  1. A/B testing is a simple idea that can be simple to apply
  2. Useful for more than incremental optimization — A/B tests can yield deep insight
  3. Fail Fast and Just Test It — A/B tests have the highest ROI of any data activity

Andrew worked for both Disney and Tinder building the A/B test to improve usability and overall growth of the companies. This article takes examples from both. Including, this test from tinder determining what iconography to use.


This article also discusses some critical testing from the widely successful SpaceX. Don’t miss this great article from Medium.

In Summary

As a takeaway- a collection of quotes from the ones who have gotten it right!

Jeff Bezos, Amazon: “Our success at Amazon is a function of how many experiments we do per year, per month, per week, per day.”

Google: “… Experimentation is practically a mantra; we evaluate almost every change that potentially affects what our users experience.”

Netflix: “… Every product change Netflix considers goes through a rigorous A/B testing process before becoming the default user experience.”

We hope this sparked some ideas for you. Remember, if we are not testing and improving, we are not growing. What areas of your site could use a re-vamp?

See you next month!

Are you interested in a free conversion optimization audit?

Submit your site and let us send you a list of custom optimizations just for you!

Claim your Free Audit Here.

Author: Kurt Philip


Submit a Comment

Your email address will not be published. Required fields are marked *

Hi, I’m Kurt Philip, the founder & CEO of Convertica. I live and breathe conversion rate optimization. I hope you enjoy our findings.

We’ve worked with over 1000 businesses in the last 6 years.
Let’s jump on a quick call to see how we can help yours.

Book FREE CRO Call

Client Case Studies

Follow us on Youtube