What confidence level should I use?

95% is the standard in most industries. Use 90% when quick decisions matter more than precision, and 99% when a bad call is very costly (e.g., pricing changes or surgical UI changes affecting millions of users).

The p-value is the probability of seeing a difference this large — or larger — purely by chance, assuming the two variants actually perform identically. A p-value below 0.05 means there is less than a 5% chance the result is random noise (at 95% confidence).

How many visitors do I need before running this test?

You need enough visitors for your sample to be representative. As a rule of thumb, aim for at least 100 conversions per variant before drawing conclusions. Use a sample-size calculator before launching the test.

Does this handle one-tailed or two-tailed tests?

This calculator performs a two-tailed z-test, which is the safer default. It tests whether the variant differs from control in either direction, not just whether it is better.

Why might a large uplift still be non-significant?

A large percentage uplift on a small sample has wide confidence intervals, so the result could easily be noise. Statistical significance depends on both effect size and sample size.

A/B Test Statistical Significance Calculator

Enter visitors and conversions for control and variant to find out if your A/B test result is statistically significant at 90%, 95%, or 99% confidence.

Test Parameters

Confidence Level

Control (A)

Visitors

Conversions

Variant (B)

Visitors

Conversions

Fill in visitor and conversion counts,
then click Calculate Significance.

Summary

This calculator runs a two-proportion z-test on your A/B test data. Paste in visitors and conversions for both variants, choose a confidence level, and see instantly whether the difference in conversion rates is statistically significant — along with relative uplift, p-value, and a plain-English verdict.

How it works

Enter the number of visitors and conversions for your control group.
Enter the number of visitors and conversions for your variant group.
Choose a confidence level: 90%, 95%, or 99%.
Click "Calculate" to run a two-proportion z-test.
Read the result: significant or not, plus p-value, z-score, and relative uplift.

Use cases

Determine whether a landing page redesign lifts conversion rates meaningfully.
Validate email subject line tests before rolling out the winner to your full list.
Check if a new CTA color or copy change moves the needle beyond random chance.
Decide when to call an A/B test and stop collecting data.
Present test results to stakeholders with a confidence score they can act on.

Frequently Asked Questions

Last updated: 2026-06-19 · Reviewed by Nham Vu