Home / Marketing / A/B Test Significance Calculator
A/B Test Significance Calculator
Enter visitors and conversions for A and B to find out whether the difference is statistically significant — or just noise.
| Visitors | Conversions | Conversion rate | |
|---|---|---|---|
| A (control) | – | ||
| B (variant) | – |
Method: a two-proportion z-test with pooled standard error, the standard approach for conversion-rate tests. Decide your sample size before starting and don’t stop the test the moment it turns significant — “peeking” inflates false positives. Run tests for at least a full business cycle (usually one or two weeks) to smooth out weekday effects.
About the A/B Test Significance Calculator
The A/B Test Calculator tells you whether the difference between two versions of a page, email or ad is statistically significant. Enter visitors and conversions for the control and the variant to see each conversion rate, the relative uplift, the p-value and z-score, at 90%, 95% or 99% confidence, with a one- or two-sided test.
How to use the A/B Test Significance Calculator
- Enter visitors and conversions for version A (the control).
- Enter visitors and conversions for version B (the variant).
- Choose the confidence level and hypothesis type.
- Read the uplift, p-value and whether the result is significant.
Frequently asked questions
What does statistically significant mean?
It means the difference is unlikely to be down to chance. At 95% confidence there is less than a 5% probability that a difference this big would appear if the versions actually performed the same.
What is a p-value?
The probability of seeing a result at least this extreme if there were really no difference. A p-value below 0.05 is significant at 95% confidence.
How long should I run an A/B test?
Until you reach the sample size you planned, and for at least one or two full weeks to cover weekday and weekend behaviour. Stopping the moment a result looks significant leads to false wins.