Skip to the content

A/B Test Sample Size Calculator

How many visitors each version of an A/B test needs before it can detect the improvement you are hoping for.

%
%

Relative: takes 5% to 5.5%

%

Chance of finding an uplift that is real

Confidence
Test

Visitors needed per variant

31,234

62,468 in total for A and B. With that many, a real change from 5% to 5.5% is found in 80 out of 100 tests at 95% confidence (two-sided).

Share this tool:TwitterFacebookLinkedInWhatsAppEmail

How many visitors an A/B test needs

Three choices fix the size of a test before it starts: your current conversion rate, the smallest improvement you would act on, and how sure you want to be. The calculator turns them into a number of visitors per variant. With two variants, A and B, the test needs twice that in total.

Current rateUplift of 5%Uplift of 10%Uplift of 20%Uplift of 50%
1%637,010163,09542,6937,750
2%315,20680,68221,1093,826
5%122,12431,2348,1581,471
10%57,76314,7513,841686
20%25,5836,5101,683294

The table is for 95% confidence, two-sided, and 80% power. Along a row, halving the uplift takes about four times the visitors. Down a column, a lower conversion rate needs more traffic for the same relative uplift.

The three inputs

Current conversion rate. From the same page and the same kind of traffic the test will get, over a few ordinary weeks.

Smallest uplift worth detecting. Relative to the current rate: 10% takes a rate of 5% to 5.5%. Choose the smallest gain that would pay for making the change. Anything smaller the test is not built to find.

Power. The chance of finding an uplift of that size when it is really there. At 80%, one real improvement in five goes unnoticed. Going to 90% takes about a third more visitors.

From visitors to days

Divide the visitors per variant by what each variant gets in a day. With 2,000 visitors a day, split evenly, each variant gets 1,000, so the 31,234 of the example take 32 days. Round up to whole weeks, so that weekdays and weekends count equally. If that runs to many months, test a bolder change or a busier page.

Then leave it alone

The number only protects you if you wait for it. Stopping when the result first looks significant, or running on because it is nearly there, both push the share of false winners above the level you chose.

The A/B test calculator page explains the test behind both tabs, the formulas and where they fall short.