Question 1

How do I read the results table?

Accepted Answer

Each row is one metric. You see A and B, their ranges (confidence intervals), the difference, the p-value, and whether it is significant. Highlighted rows mean A and B differ.

Question 2

What do the confidence intervals mean?

Accepted Answer

They are the 'likely range' around a number. At 95%, the true value should fall in that range most of the time. If the difference range does not cross 0, we mark it significant.

Question 3

What does the p-value represent?

Accepted Answer

It tells how surprising the result would be if there were no real difference. Smaller than 0.05 (for a 95% level) means it is statistically significant.

Question 4

What is Rel. Change and the mini error bar?

Accepted Answer

Rel. Change is the percent change of B vs A. The small bar shows its range; colored = significant, gray = not.

Question 5

Can I use unequal sample sizes for A and B?

Accepted Answer

Yes. Different sample sizes are fine. Equal sizes usually give more power.

Question 6

When should I use proportion vs. continuous metrics?

Accepted Answer

Proportion: yes/no outcomes (e.g., converted). Continuous: numbers (e.g., revenue, time). For continuous, enter mean and standard deviation for each group.

Metric	Type	Group A (CI)	Group B (CI)	Difference (CI)	Rel. Change	P-Value	Result
Adoption Rate	proportion	10.00% (8.14% – 11.86%)	12.00% (9.99% – 14.01%)	2.00% (-0.74% – 4.74%)	20.00%	0.1529	Not Significant
Spend per User	continuous	25.00 (24.50 – 25.50)	28.00 (27.50 – 28.50)	3.00 (2.30 – 3.70)	12.00%	0.0000	B higher Significant

Metric	Type	Group A (CI)	Group B (CI)	Difference (CI)	Rel. Change	P-Value	Result
Adoption Rate	proportion	10.00% (8.14% – 11.86%)	12.00% (9.99% – 14.01%)	2.00% (-0.74% – 4.74%)	20.00%	0.1529	Not Significant
Spend per User	continuous	25.00 (24.50 – 25.50)	28.00 (27.50 – 28.50)	3.00 (2.30 – 3.70)	12.00%	0.0000	B higher Significant

A/B Test Results Visualizer

Metrics

Frequently asked questions