A/B testing tools

What are A/B testing tools?

A/B testing tools are software that shows different versions of a web page, app screen or feature to randomly chosen visitors, keeps each visitor in the same version, tracks the goals you set, and reports whether the difference between versions is bigger than chance. Most also let you target segments and test more than two versions. The tool runs the test; good conversion testing also needs research, a hypothesis and a sample size set in advance.

They fall into two groups. Web testing tools are built for marketers: a script on the page and a visual editor to change headlines, layouts and calls to action. Product experimentation platforms are built for developers: experiments run in code, behind feature flags, on websites, servers and apps. Many tools now do some of both.

How do A/B testing tools work?

An A/B testing tool works in five steps: it assigns each visitor to a version at random, remembers that assignment, shows the version, records the goals you defined, and compares the groups with a statistical test. The tool handles the mechanics; you supply the hypothesis, the variant and the sample size.

1. Assignment: a script in the browser, or an SDK on your server or in your app, puts each visitor into the control or a variant.
2. Persistence: a cookie, user ID or hash keeps that visitor in the same version on every visit, so results are not mixed.
3. Rendering: client-side tools change the page after it loads; server-side tools send the chosen version directly.
4. Tracking: the tool records goals such as clicks, form submissions, orders or revenue, often through your analytics.
5. Analysis: the stats engine reports the difference between versions and how likely it is to be real rather than noise.

Which A/B testing tool fits your situation?

The right A/B testing tool depends more on your team and what you test than on any ranking. This table matches common situations to the tools on this page whose own sites describe that use. It is a starting shortlist, not a recommendation of one vendor over another.

Best A/B testing tools for product and engineering teams

Product experimentation platforms run tests in code, usually behind feature flags, so they can test anything a developer can build: pricing logic, search results, onboarding flows and app screens. They tend to offer stronger statistics and analysis in your own data warehouse, and they need developer time for every test.

Statsig

A product experimentation platform with feature flags, analytics and session replay.

• A stats engine with variance reduction and sequential testing
• Warehouse-native or hosted deployment
• Feature flags, product analytics, session replay and web analytics
• Switchback tests and multi-armed bandits

Suits: Product and engineering teams running many experiments. Statistics named: Sequential, with variance reduction. Free plan: Free Developer tier stated, with A/B tests, feature flags and analytics. Vendor site

GrowthBook

An open source experimentation and feature flag platform that can be self-hosted.

• Open source, with a self-hosted option
• Warehouse-native analysis with Bayesian and sequential statistics
• Feature flags with ramp schedules and automatic rollbacks
• A visual editor and SDKs for server, client, mobile and edge

Suits: Teams that want open source, self-hosting or analysis in their own data warehouse. Statistics named: Bayesian and sequential. Free plan: Free cloud Starter plan (up to 3 users) and free self-hosted open source stated. Vendor site

PostHog

An open source product platform that combines analytics, experiments, feature flags, session replay and heatmaps.

• A/B, A/B/n, holdout and redirect tests, built on feature flags
• Bayesian or frequentist analysis
• Heatmaps, clickmaps and scrollmaps, with rage and dead click detection
• Session replay on web and in iOS and Android apps

Suits: Product teams that want testing, analytics and replay in one open source platform. Statistics named: Bayesian or frequentist. Free plan: Free tier stated, open source. Vendor site

LaunchDarkly

A feature management platform that runs experiments inside its feature flag workflow.

• Experiments on feature flags, with holdouts and mutual exclusion
• CUPED, sequential testing and sample ratio mismatch detection
• Results segmented by device, location or cohort
• Warehouse-native analysis and tests of AI prompts and models

Suits: Engineering teams who already release features behind flags. Statistics named: Sequential, with CUPED. Free plan: Free developer plan stated. Vendor site

Amplitude Experiment

Amplitude's feature experimentation product, connected to its product analytics.

• Guided experiment design with feature flags
• Client-side or server-side deployment
• Launch experiments from analytics charts and session replays
• Sequential testing, bandits, holdouts and mutual exclusion groups

Suits: Teams already using Amplitude for product analytics. Statistics named: Sequential. Free plan: Free plan stated, includes A/B tests. Vendor site

Firebase A/B Testing

Google's A/B testing for mobile apps, run through Firebase Remote Config and notifications.

• Tests of app interface, features and messaging campaigns
• Retention, revenue and engagement tracked out of the box
• Further metrics through Google Analytics

Suits: Mobile app teams on Firebase. Free plan: Stated as a no cost tool within Firebase. Vendor site

Client-side vs server-side A/B testing tools

Client-side A/B testing changes the page in the visitor's browser after it loads, so a marketer can build a test in a visual editor. Server-side A/B testing picks the version in your server or app code before anything is shown, so there is no flicker and you can test deeper logic, but every test needs a developer.

Optimizely, Kameleoon, Convert, Zoho PageSense and PostHog offer both. If your tests are mostly page changes, start client-side. If your best ideas need code, choose a tool that supports server-side tests from the start.

How to choose an A/B testing tool

Choose an A/B testing tool by who will build the tests, what you need to test, how the tool does its statistics, how it affects page speed, and whether it fits your analytics and consent setup. Price matters too, but a cheap tool your team cannot use, or one that slows your pages, costs more in the end.

• Who builds tests: marketers in a visual editor, developers in code, or both
• Test types you need: A/B, split URL, multivariate, server-side, mobile app
• Statistics: the method is documented, and it handles checking results early (sequential testing) if you plan to
• Sample ratio mismatch detection, so a broken traffic split is flagged
• No visible flicker, and a small effect on page speed
• Goals that match your business: orders, revenue, leads, not only clicks
• Integrations with your analytics, such as Google Analytics 4, and your data warehouse if you use one
• Consent and privacy controls that fit the rules where your visitors live
• Heatmaps or recordings per variant, or an easy link to the tool that has them
• Who owns the account and the test history if you change agency or consultant

Run an A/A test, two identical versions, when you first install a tool. If it reports a significant difference between identical pages, something in the setup is wrong.

Mistakes to avoid with any A/B testing tool

• Buying before checking traffic. If a test would need months to reach its sample size, no tool will make it faster.
• Stopping at the first winner. Checking a frequentist test every day and stopping when it looks significant inflates false positives.
• Measuring clicks instead of the business goal. A variant that wins on clicks can lose on orders or revenue.
• Ignoring flicker and speed. A visible flash of the original page can bias a test against the variant.
• Testing guesses. Tests built on research, such as analytics, heatmaps, recordings and surveys, win more often than tests built on opinions.
• Leaving tests running. Google's guidance on website testing says to run tests only as long as needed, then remove the variants and the test code.

How to set up your first test in any tool

1. Pick one page with enough traffic and one goal that matters, such as orders or leads.
2. Write a hypothesis from evidence: what you will change, for whom, and why you expect it to help.
3. Calculate the sample size and the number of weeks it needs.
4. Install the tool, check page speed, and run an A/A test.
5. Build the variant, check it on desktop and mobile, then launch.
6. Wait for the planned sample, for at least one full week, then read the result and check significance.

Mobile app and SEO A/B testing tools

Mobile app A/B testing needs a tool with an SDK for your app. Firebase A/B Testing, Statsig, PostHog, LaunchDarkly, Amplitude Experiment and Optimizely Feature Management test in apps, and Wingify, Convert, Kameleoon and Zoho PageSense list mobile app testing too. App tests usually change configuration or features remotely rather than redrawing a screen.

SEO A/B testing is a different method. Instead of splitting visitors, it splits a group of similar pages, changes one half, and compares organic search traffic over time. The tools on this page split visitors, so they are not built for SEO tests.

Visitor tests do not hurt search rankings when they are run properly. Google's guidance on website testing says not to cloak, to point variant URLs at the original with rel=canonical, to use temporary 302 redirects rather than 301s, and to end tests once they reach a conclusion.

Questions about A/B testing tools