Two-proportion z-test

Compares two independent proportions, using the pooled proportion for the test and the unpooled standard error for the confidence interval.

z = \frac{\phat_1 - \phat_2}{\sqrt{\bar p(1-\bar p)\left(\frac{1}{n_1} + \frac{1}{n_2}\right)}},\qquad \bar p = \frac{x_1 + x_2}{n_1 + n_2}

Test of H₀: p₁ = p₂ (pooled SE, because H₀ says both groups share one proportion).

$x_i, n_i$: Successes and trials in group i.
$\phat_i$: $x_i/n_i$ .
$\bar p$: Pooled proportion.

(\phat_1 - \phat_2) \pm z_{1-\alpha/2}\sqrt{\frac{\phat_1(1-\phat_1)}{n_1} + \frac{\phat_2(1-\phat_2)}{n_2}}

Wald interval for the difference (unpooled SE). Newcombe's hybrid-score interval is the better choice for small samples.

$z_{1-\alpha/2}$: Standard normal quantile (1.96 for 95%).

When to use

Two independent groups with a binary outcome: conversion in A vs B, default rate in two segments, recovery under two treatments. Without continuity correction, $z^2$ equals the Pearson chi-square statistic of the 2×2 table.

Assumptions

Independent observations, independent groups (random assignment or random samples).
At least about 10 successes and 10 failures in each group; otherwise use Fisher's exact test.

How to interpret

Report the difference in percentage points with its CI, the relative lift ( $\phat_1/\phat_2 - 1$ ), and the p-value. The pooled test and unpooled interval can disagree slightly at the margin of significance.

Common mistakes

Mixing up percentage points and percent ("from 12% to 15% is a 3-point or 25% increase").
Treating repeated visits by the same user as independent trials.
Peeking at the p-value daily and stopping when it dips below 0.05 (see peeking).

Example

Variant: 150 conversions of 1000; control: 120 of 1000. $\phat_1 - \phat_2 = 0.03$ , $\bar p = 0.135$ , $z = 1.96$ , $p = 0.0496$ ; 95% CI for the difference 0.0001 to 0.0599 (0.01 to 6.0 points). The evidence of a lift is borderline, and its size is very uncertain.