Sample Size Calculator for Two Proportions

What Is Sample Size Calculation for Proportions?

When comparing two success rates (conversion rates, cure rates, pass rates), you need enough subjects to reliably detect a difference. This calculator determines sample size for comparing two proportions.

Input:

• Baseline proportion (p₁)
• Expected proportion after treatment (p₂)
• Significance level (α)
• Desired power

Output: Required sample size per group

The Formula

For a two-proportion z-test with equal groups:

$n \approx \frac{\left[z_{1-\alpha/2} \sqrt{2\bar{p}(1-\bar{p})} + z_{1-\beta} \sqrt{p_1(1-p_1) + p_2(1-p_2)}\right]^2}{(p_1 - p_2)^2}$

Where $\bar{p} = (p_1 + p_2)/2$ is the average proportion.

This is simplified for practical calculation—the calculator handles the complexity.

Practical Examples

Clinical Trial

"Does the new treatment increase cure rate from 60% to 75%?"

p₁ = 0.60, p₂ = 0.75, difference = 15 percentage points α = 0.05, power = 0.80

n ≈ 120 per group

Marketing A/B Test

"Does the new landing page improve conversion from 4% to 5%?"

p₁ = 0.04, p₂ = 0.05, difference = 1 percentage point α = 0.05, power = 0.80

n ≈ 3,500 per group

Notice: Small absolute differences in proportions near 0 or 1 require very large samples.

Education Research

"Does the intervention improve pass rates from 70% to 80%?"

p₁ = 0.70, p₂ = 0.80, difference = 10 percentage points α = 0.05, power = 0.80

n ≈ 200 per group

Key Factors Affecting Sample Size

Baseline Proportion

Proportions near 0.5 have maximum variance, requiring larger samples.

Same absolute difference, different sample requirements.

Effect Size (Difference)

Smaller differences require larger samples:

Power and Significance Level

Higher standards require more subjects.

One-Sided vs. Two-Sided

Two-sided (default): Tests whether proportions differ in either direction.

One-sided: Tests whether p₂ > p₁ specifically. Requires fewer subjects but only appropriate with strong prior justification.

Unequal Allocation

Sometimes unequal groups make sense:

• Control is cheap, treatment is expensive
• Ethical constraints
• Practical limitations

For allocation ratio r = n₂/n₁:

Optimal for efficiency: r = 1 (equal groups) If r ≠ 1, total sample increases.

Practical Considerations

Attrition

Plan for dropout:

n_recruited = n_needed / (1 - dropout_rate)

Minimum Cell Counts

Ensure expected successes and failures meet test assumptions:

• n × p ≥ 5
• n × (1-p) ≥ 5

Very small or very large proportions may require exact tests.

Clustering

If subjects aren't independent (students in classrooms, patients in clinics), standard formulas underestimate required n. Apply design effects.

Relative vs. Absolute Difference

Absolute difference: p₂ - p₁ "Improved from 10% to 15%" = 5 percentage point improvement

Relative difference: (p₂ - p₁)/p₁ "50% relative improvement" (10% to 15%)

Sample size depends on absolute difference. The same relative improvement requires vastly different samples depending on baseline:

• 1% to 1.5% (50% relative): n ≈ 5,400 per group
• 40% to 60% (50% relative): n ≈ 85 per group

Historical Context

Sample size formulas for proportions were developed alongside hypothesis testing frameworks in the early 20th century.

Joseph Fleiss provided comprehensive formulas in his influential textbook "Statistical Methods for Rates and Proportions" (1973).

The development of A/B testing platforms has made these calculations routine in product development and marketing.

Common Mistakes

Using Relative Instead of Absolute

A "20% improvement" could mean 10% to 12% (2pp) or 50% to 60% (10pp). Clarify which.

Ignoring Baseline Variability

Proportions near 0 or 1 have less room to move, affecting calculations.

Not Accounting for Multiple Tests

If running many tests, adjust for multiple comparisons or increase individual test power.

Related Calculations

• Sample Size for Two Means — For continuous outcomes
• A/B Test Sample Size Calculator — Specialized for web testing
• Statistical Power Calculator — Given n, find power
• CI for Difference in Proportions — Analyze results
• Odds Ratio Calculator — Alternative effect measure

Summary

Sample size for comparing proportions depends on baseline rate, expected difference, significance level, and power. Small differences and proportions near 50% require the largest samples. The calculator handles the complex formula, but understanding the inputs—realistic effect sizes, appropriate significance levels, and adequate power—is essential for planning studies that can detect meaningful differences when they exist.