Calculate the sample size required for your next A/B test
A/B Test Sample Size Calculator
Find out how many visitors you need to reach statistical significance.
%
%
%
%
Sample size per variation
—visitors
Total sample size
—
Test duration
—
Conversions needed
—
FAQ
Frequently asked questions
How many visitors do I need for an A/B test?
It depends on your baseline conversion rate and how small a change you want to detect. For a typical SaaS page with a 3% baseline and a 20% relative MDE target, expect around 4,000 visitors per variation at 95% confidence and 80% power. Use the calculator above for your exact numbers.
What is statistical power and why does it matter?
Power is the chance your test will detect a real effect if one exists. At 80% power, if the variation really is better, you have an 80% chance of catching it with this sample size. Lower power means more missed winners.
One-sided or two-sided test?
Two-sided is the safe choice. Use one-sided only when a worse-than-control result would be just as actionable as no result (rare). One-sided cuts sample size by roughly 20%, but it's easy to misuse.
What happens if I check my test results too early?
The earlier you check, the higher your false positive rate. Tests checked daily and stopped at the first sign of significance produce wrong answers about 30% of the time, even at 95% confidence. Set the sample size, then check only once you hit it.
What is statistical significance in A/B testing?
Statistical significance means the difference you observed is unlikely to be caused by random chance. At 95% confidence, there's a 5% chance the result is a fluke. The lower the chance of a fluke, the more confident you can be that the change caused the lift.
Should I use relative or absolute MDE?
Relative is more intuitive for most marketing teams. A 10% relative lift on a 3% baseline means going from 3% to 3.3%. Absolute would mean going from 3% to 13%, which is rarely realistic.
Can I run an A/B test with low traffic?
Yes, but you need to either accept a larger MDE (only detect bigger lifts), test higher-leverage pages (checkout vs blog), or run the test longer. Sub-1,000 daily visitors means most tests will take 4 to 8 weeks.
Why does the calculator show a different result than Optimizely's?
Optimizely's calculator uses a sequential testing method with a different assumption (power approaching 1). Our calculator uses the classical fixed-horizon formula that Evan Miller's calculator uses, which is the standard reference.
