Case study
Affiliate income case study, part 2: about 50 split tests and $32,532 more monthly income in under 90 days
This affiliate income case study is part 2 of the story of Robert's affiliate site. After the first month's quick wins, Convertica ran split test after split test on the site for three months. The original write-up reports monthly income up $32,532 in under 90 days, and the lessons are less about any single test than about how to run many of them without letting hope or fear make the calls.
Affiliate income case study at a glance
| Client | Robert, owner of a large affiliate review site (site kept private; the same site as part 1) |
|---|---|
| Vertical | Affiliate |
| Traffic | Over 200,000 unique views a month |
| Period | About three months, written up in November 2017 |
| Tests run | About 50 split tests (the original says "over 50" in one place and "closer to 50" in another) |
| Revenue | Up by over 100% for this site over the period, as stated |
| Tools named | Hotjar, VWO, Google Calendar, Slack |
+$32,532
Monthly income, Robert's affiliate site, under 90 days
Source: Robert's affiliate site, as stated in the original case study headline (calculation not published)
What happened after part 1?
Part 1 of this affiliate case study ended with Robert's site earning an extra $15,048 in monthly revenue in August 2017, from fixing the basics where visitors clicked. Part 2 is what came next: three months of split testing, one test after another, on a site with enough traffic to learn fast.
The original write-up opens with a phone call. Three years earlier, Robert had set himself an ambitious monthly income goal across all his sites. As Kurt Philip was packing for a conference in Chiang Mai, Robert called to say he had hit it. This site was one part of his portfolio. According to the original case study, its revenue rose by over 100% over the three months, and the headline puts the gain in monthly income at $32,532 in under 90 days. The calculation behind that figure was not published.
What is a CRO baseline?
A CRO baseline is the set of conversion rate optimization best practices a site should have in place before split testing begins. Testing small ideas on a page with obvious problems wastes traffic, so fix the obvious first, then test. For an affiliate site, the original case study's baseline has four parts:
- High-quality images of the products you promote.
- Clear calls to action. Buttons, not buried text links.
- Features and benefits shown so readers can make an informed decision.
- Every element working on mobile. The original notes that a surprising number of the sites Convertica audited at the time did not display properly on a phone.
With the baseline in place, the next job is a long list of experiments to run. That is why an owner who has never done CRO is good news: the list is full of untested ideas.
How much can one small change move clicks?
In a test from the month the original was written, moving the calls to action from the center of a product box to the left, under the image, increased clicks to the calls to action by 23.5%. The original does not say which site it ran on, and it is the kind of result that sounds too easy, which is exactly why it has to be tested on your own page rather than copied.
Moving a button will not work the same way on every website. Convertica does not start from scratch on each site either: it keeps a growing list of layouts, buttons and combinations that have won on similar sites, and tests them. The list makes good first guesses more likely. It never replaces the test.
Why does early split test data mislead?
Early split test data misleads because it rests on a handful of conversions, so the range of possible results is huge. The original walks through a scenario every site owner will recognize: you start a test, check it an hour later, and the testing tool shows a possible gain of several hundred percent.
With 4 conversions on one version and 9 on the other, VWO put the possible change anywhere from -41.1% to +641.9%. In the story, the owner sees only the top of that range and starts planning how to spend the money. By the next morning, more data has arrived and the range has narrowed to about -10% to +32%. The test eventually finished at around a 30% lift in click-through rate: a good result, nowhere near the first hour's dream.
The fix is to decide the sample size before the test starts and not call it before then. The A/B test sample size calculator gives that number, and the statistical significance calculator checks the result when the test ends.
How do split test wins compound?
Split test wins compound because each new test runs against the last winner, so the gains multiply rather than add. Two modest wins in a row can produce a large total, which is why a program of many tests beats the search for one big one. The original case study uses a simple example, starting from 100 clicks:
| Stage | Lift in click rate | Multiplier | Clicks |
|---|---|---|---|
| Original | None | 1.000 | 100 |
| After split test 1 | +23.6% | 1.236 | 123.6 |
| After split test 2 | +6.5% | 1.065 | 131.6 (+31.6% in total) |
Two corrections to the original, for honesty. It gives the first lift as 23.5% in one place and 23.6% in another. And it reports the total as 32.6%, because it rounded the first step to 124 clicks before applying the second (124 x 1.065 = 132). Without the rounding the total is 31.6%. The point stands either way: the second, smaller win builds on the first.
How much traffic does split testing need?
Split testing needs enough traffic to reach a reliable answer in a sensible time, and the more traffic a site has, the more tests it can run and the faster it learns. The original case study makes the point with a small example site, which this page calls Site A:
| Detail | Site A | Robert's site |
|---|---|---|
| Traffic | 5,000 visitors a month; 2,300 to the busiest page | Over 200,000 unique views a month |
| Test | Button text "View Price" against "Check Price", one page | About 50 tests across the site in three months |
| Visitors per version | About 575 each over two weeks | Enough to run several tests a week |
| Outcome | The variation lost by 10% after two weeks of waiting | Winners found fast, kept and built on |
On Site A, two weeks of waiting for a result that turns out to be a loss feels like a waste, and that is where many owners give up on testing. It is not a reason to stop, but it is a reason to choose tests carefully: with little traffic, test only changes big enough to show up, on the pages that earn. At the time, Convertica focused each campaign on a client's top five pages, which in most cases made up most of the site's revenue.
Why do emotions get in the way of split testing?
Emotions get in the way of split testing because people want to be right, and money brings out greed and fear. Greed reads the top of an early range as the result. Fear calls a test off after one bad day. Both lead to decisions the data does not support, and the bigger the site's income, the stronger the pull.
The original case study's advice is simple and hard: expect nothing. A split test is variation A against variation B, nothing more. An outside team has an advantage here, because it did not build the site and has no history with the page, so it finds it easier to let the numbers decide.
How do you run 50 split tests without losing track?
You run 50 split tests without losing track by following one repeatable process: a plan for what to test, a tool for each job, documented results for every test, and someone managing each campaign. At Robert's scale, testing without that would have been impossible to manage.
- Keep a list of experiments, ranked by the pages and products that earn most.
- Use one tool per job: heatmaps and recordings (the original used Hotjar, now part of Contentsquare, which no longer opens new Hotjar accounts), split testing (VWO), a schedule (Google Calendar) and one place to talk (Slack).
- Document every test, winners and losers, so the list of what works on similar sites grows.
- Run tests back to back against the latest winner, so the gains compound.
What this case study does not show
The original write-up does not publish the individual tests, their dates or their results, apart from the examples above. It does not show how the $32,532 figure was calculated or which months it compares, and its count of tests varies between "over 50" and "closer to 50". The Site A numbers are an illustration, not a client. Read the headline as the reported outcome of a long testing program on a high-traffic site, not as what a single change will do.
What can you apply to your own affiliate site?
You can apply this affiliate income case study to any site with enough traffic to test: get the baseline right, keep a ranked list of experiments, run them back to back against the latest winner, and set the rules for calling a test before it starts.
- Fix the baseline before you test. Images, clear buttons, features and benefits, and a page that works on a phone.
- Test ideas that worked elsewhere, but never assume they will work on your site.
- Plan the sample size first. Do not read anything into the first hour, or the first day.
- Let wins compound. Each new test runs against the current winner.
- Match the tests to your traffic. Small sites should test fewer, bigger changes on their top pages.
- Write down every result. The record is what turns 50 tests into a method.
One addition for 2026: before you test for people, check what AI agents and AI search see. A page whose top pick, prices and verdict are readable text serves both. AI agent readiness is one of the eight checks in Convertica's free CRO audit.
Start with part 1 of this affiliate case study if you have not read it. For a later affiliate campaign with every test measured in Amazon revenue, read the 10Beasts case study, and for the test setup Convertica used on affiliate sites, read why Amazon affiliate split tests fail.
Affiliate income case study FAQ
What does this affiliate income case study cover?
It covers three months of split testing on Robert's affiliate site, the same site as part 1. Convertica ran about 50 split tests back to back, kept the winners and built on them. The original write-up reports monthly income up $32,532 in under 90 days and says revenue for the site more than doubled over the period.
How was the $32,532 figure calculated?
The original case study gives $32,532 in its headline as the increase in monthly income in under 90 days, but does not publish the calculation or the months it compares. In the text it says revenue for this site rose by over 100% over the three months. Both are shown on this page as stated.
Why does early split test data look so dramatic?
Because it rests on very few conversions. An hour into one test, VWO showed a possible change of -41.1% to +641.9%, with 4 conversions on one version and 9 on the other. A range that wide means the test knows almost nothing yet. Wait for the sample size you planned before reading a result.
How do split test wins compound?
Each new test is measured against the last winner, so the gains multiply. A 23.6% lift followed by a 6.5% lift gives 1.236 x 1.065 = 1.316, a 31.6% total gain over the original. The original write-up says 32.6% because it rounded the first step to 124 clicks before applying the second.
How much traffic do you need to split test an affiliate site?
Enough to reach a reliable answer in a sensible time. On a page with 2,300 visits a month, a two-week test gives each version about 575 visitors, which is only enough to detect a large difference. With over 200,000 unique views a month, Robert's site could run several tests a week.
Why do emotions get in the way of split testing?
Because people want to be right, and money brings out greed and fear. Owners tend to celebrate early results that look like a big win and feel cheated by a test that loses after two weeks. Running tests to a plan, and expecting nothing from any one variation, keeps decisions on the data.
What tools were used in this affiliate case study?
The original write-up lists Hotjar for heatmaps and recordings, VWO for split testing, Google Calendar for scheduling and Slack for communication. The tools matter less than the process: a plan, documented results and someone managing each test.