Two Proportion Z-Test Calculator

What Is a Two-Proportion Z-Test?

The two-proportion z-test determines whether two population proportions differ significantly from each other.

Question answered: "Do these two groups have different rates/proportions?"

Examples:

• "Is the conversion rate different between website variants A and B?"
• "Do men and women have different voting preferences?"
• "Is the success rate higher for the new treatment versus standard?"

The Hypotheses

Null hypothesis (H₀): p₁ = p₂ (proportions are equal)

Alternative hypothesis (H₁):

• Two-tailed: p₁ ≠ p₂ (proportions differ)
• Right-tailed: p₁ > p₂ (first proportion is greater)
• Left-tailed: p₁ < p₂ (first proportion is less)

The Test Statistic

$z = \frac{\hat{p}_1 - \hat{p}_2}{\sqrt{\hat{p}(1-\hat{p})\left(\frac{1}{n_1} + \frac{1}{n_2}\right)}}$

Where:

• $\hat{p}_1 = x_1/n_1$ (sample proportion 1)
• $\hat{p}_2 = x_2/n_2$ (sample proportion 2)
• $\hat{p} = (x_1 + x_2)/(n_1 + n_2)$ (pooled proportion)

The pooled proportion assumes $H_0$ is true (both populations have the same rate).

Example Calculation

Scenario: A/B test comparing two landing pages.

Data:

• Page A: 45 conversions in 1,200 visitors (p̂₁ = 0.0375)
• Page B: 72 conversions in 1,500 visitors (p̂₂ = 0.0480)

Pooled proportion: p̂ = (45 + 72) / (1,200 + 1,500) = 117/2,700 = 0.0433

Standard error: SE = √[0.0433 × 0.9567 × (1/1200 + 1/1500)] SE = √[0.0414 × 0.00150] = √0.0000621 = 0.00788

Test statistic: z = (0.0375 - 0.0480) / 0.00788 = -1.33

P-value: For two-tailed test, p = 0.183

Conclusion: No significant difference between pages (p > 0.05).

Pooled vs. Unpooled Standard Error

Pooled (for hypothesis testing)

Uses combined proportion assuming $H_0$ is true:

$SE_{\text{pooled}} = \sqrt{\hat{p}(1-\hat{p})\left(\frac{1}{n_1} + \frac{1}{n_2}\right)}$

Unpooled (for confidence intervals)

Uses separate sample proportions:

$SE_{\text{unpooled}} = \sqrt{\frac{\hat{p}_1(1-\hat{p}_1)}{n_1} + \frac{\hat{p}_2(1-\hat{p}_2)}{n_2}}$

Why different? Hypothesis test assumes equal proportions; confidence interval estimates the actual difference.

Confidence Interval for Difference

$CI = (\hat{p}_1 - \hat{p}_2) \pm z^* \times SE_{\text{unpooled}}$

For 95% CI ($z^* = 1.96$):

$CI = (0.0375 - 0.0480) \pm 1.96 \times SE_{\text{unpooled}} = -0.0105 \pm 0.0154 = (-0.0259, 0.0049)$

The interval includes 0, consistent with non-significant test result.

Assumptions

Independence

• Observations within each group are independent
• Groups are independent of each other
• No subject appears in both groups

Adequate Sample Size

Both groups should have:

• n₁p̂₁ ≥ 10 and n₁(1-p̂₁) ≥ 10
• n₂p̂₂ ≥ 10 and n₂(1-p̂₂) ≥ 10

If violated: Use Fisher's Exact Test.

Random Sampling

Samples represent their respective populations.

Applications

A/B Testing

The classic use case: comparing conversion rates between variants.

• Website versions
• Email subject lines
• Ad creatives
• Pricing strategies

Clinical Trials

Comparing treatment success rates.

• Drug vs. placebo
• New treatment vs. standard
• Different dosages

Market Research

Comparing preferences or behaviors between segments.

• Brand awareness by demographic
• Purchase intent by exposure group

Quality Control

Comparing defect rates between:

• Production lines
• Shifts
• Suppliers
• Time periods

Social Science

Comparing proportions between populations.

• Voting patterns by demographic
• Survey responses by group

One-Tailed vs. Two-Tailed

Two-tailed (default): Tests for any difference between proportions. "Are the rates different?"

One-tailed: Tests for difference in specific direction. "Is the new treatment better (not just different)?"

One-tailed requires prior justification. Don't choose after seeing results.

Effect Size: Cohen's h

$h = 2 \arcsin(\sqrt{p_1}) - 2 \arcsin(\sqrt{p_2})$

Or using the difference directly:

Risk Difference = $p_1 - p_2$ (absolute difference)

Relative Risk = $p_1/p_2$ (ratio)

Relationship to Chi-Square

For 2×2 tables:

z² = χ² (with df = 1)

The two-proportion z-test is mathematically equivalent to chi-square test of independence for 2×2 tables.

Advantage of z-test: Provides directional information (positive or negative z).

Common Mistakes

Comparing Dependent Proportions

If the same subjects are measured twice (before/after), use McNemar's test, not two-proportion z-test.

Using Unpooled SE for Hypothesis Test

The pooled SE assumes H₀ is true, which is appropriate for hypothesis testing.

Ignoring Sample Size Requirements

With small samples or extreme proportions, use Fisher's Exact Test.

Multiple Comparisons

Testing many variants inflates false positive rate. Use adjustments (Bonferroni) or control procedures.

Reporting Results

APA format:

"A two-proportion z-test indicated no significant difference in conversion rates between Page A (3.75%, n = 1,200) and Page B (4.80%, n = 1,500), z = -1.33, p = .183, 95% CI for difference [-2.59%, 0.49%]."

Include:

• Both sample proportions and sizes
• z-statistic
• P-value
• Confidence interval for difference

Sample Size Planning

For adequate power, use:

n per group = 2 × [z₁₋α/₂ × √(2p̄q̄) + z₁₋β × √(p₁q₁ + p₂q₂)]² / (p₁ - p₂)²

Use A/B test sample size calculator for planning.

Continuity Correction

For better approximation with smaller samples:

z = (|p̂₁ - p̂₂| - 1/2 × (1/n₁ + 1/n₂)) / SE

Makes the test slightly more conservative.

Related Measures

Risk Difference (Absolute)

RD = p₁ - p₂

"Treatment increases success by 5 percentage points."

Relative Risk

RR = p₁/p₂

"Treatment patients are 1.5 times more likely to succeed."

Odds Ratio

OR = [p₁/(1-p₁)] / [p₂/(1-p₂)]

"The odds of success are 2.3 times higher with treatment."

Historical Context

The two-proportion z-test developed alongside the broader framework of large-sample theory in the late 19th and early 20th centuries.

The test applies the Central Limit Theorem: sample proportions are approximately normal for large n, so their difference is also approximately normal.

With modern computing, the approximate z-test is often replaced by exact methods (Fisher's) for small samples, but remains the standard for large-sample inference about proportions.

Extensions

Stratified Analysis

When comparing proportions across strata (e.g., age groups), use Mantel-Haenszel or stratified methods.

Multiple Groups

For comparing proportions across 3+ groups, use chi-square test of independence.

Sequential Testing

In ongoing A/B tests, use sequential analysis methods to control error rates with multiple looks at data.

Related Calculations

• One-Proportion Z-Test — Single proportion vs. value
• Chi-Square Independence Test — Categorical association
• Fisher's Exact Test — Small samples
• Confidence Interval for Difference in Proportions — Interval estimate
• A/B Test Sample Size — Planning
• Relative Risk Calculator — Effect size

Summary

The two-proportion z-test determines whether proportions differ between two independent groups by calculating z = (p̂₁ - p̂₂) / SE, using pooled standard error that assumes equal proportions under H₀. The test requires adequate sample sizes (np ≥ 10 for successes and failures in both groups); otherwise use Fisher's Exact Test. Report both proportions, z-statistic, p-value, and confidence interval for the difference. The test is mathematically equivalent to chi-square for 2×2 tables but provides directional information through the sign of z.