Confidence Interval Calculator for Difference in Proportions

Calculate CI for the difference between two proportions.

Group 1

Group 2

Group 1: Successes must be 0 ≤ x ≤ n

Examples:

Enter success counts and sample sizes for both groups

What Is a Confidence Interval for the Difference in Proportions?

When comparing success rates between two groups—drug vs. placebo response, website A vs. B conversion, treatment vs. control cure rate—you want to know: how different are the proportions?

A confidence interval for (p₁ - p₂) provides a range:

"The new treatment had a 12% higher success rate (95% CI: 7% to 17%)"

This captures both the estimated difference and its precision.

The Formula

Standard (Wald) interval:

CI = (p̂₁ - p̂₂) ± z* × √[p̂₁(1-p̂₁)/n₁ + p̂₂(1-p̂₂)/n₂]

Where:

  • p̂₁, p̂₂ = sample proportions
  • n₁, n₂ = sample sizes
  • z* = critical z-value (1.96 for 95%)

Improved methods:

The Agresti-Caffo interval adds 1 success and 1 failure to each group before calculating:

p̃₁ = (x₁ + 1) / (n₁ + 2) p̃₂ = (x₂ + 1) / (n₂ + 2)

This provides better coverage, especially for small samples or extreme proportions.

Interpretation

Zero in the Interval

CI: (-3% to +5%)

  • Zero is included
  • Can't conclude proportions differ
  • Difference is not statistically significant

Zero Not in the Interval

CI: (3% to 9%)

  • Zero is excluded
  • Proportions genuinely differ
  • Difference is statistically significant

Applications

Clinical Trials

"Treatment group: 78% improved vs. Control: 62% improved" "Difference: 16 percentage points (95% CI: 10% to 22%)"

Clinicians can see both the magnitude and precision of the treatment effect.

A/B Testing

"Version A: 4.2% conversion vs. Version B: 5.1% conversion" "Difference: 0.9 percentage points (95% CI: 0.2% to 1.6%)"

The interval excludes zero, suggesting Version B genuinely performs better.

Public Health

"Vaccination rate in Region A: 72% vs. Region B: 65%" "Difference: 7 percentage points (95% CI: 3% to 11%)"

Policy makers can assess whether regional differences are real.

Survey Comparisons

"Support in 2023: 54% vs. 2024: 48%" "Difference: 6 percentage points (95% CI: 2% to 10%)"

Pollsters can claim genuine change occurred if zero is outside the interval.

Why Not Just Subtract?

Simply reporting 78% - 62% = 16% misses the uncertainty. With small samples, that 16% difference might be consistent with true differences ranging from 5% to 27%—or even including zero.

The confidence interval reveals whether the observed difference is likely real or just sampling noise.

Historical Context

Comparing proportions has roots in early probability theory. Abraham de Moivre and Pierre-Simon Laplace developed foundations in the 18th century.

The modern framework emerged with Karl Pearson's chi-square test (1900) and was refined throughout the 20th century.

Agresti and Caffo (2000) showed that simple adjustments—adding pseudo-observations—dramatically improve interval coverage. Their work demonstrated that the standard Wald interval often has actual coverage well below nominal, especially for small samples.

The Coverage Problem

Like the CI for a single proportion, the standard interval for differences has coverage issues:

  • Near extreme proportions (close to 0% or 100%)
  • With small sample sizes
  • When proportions are very different

The Agresti-Caffo adjustment—adding 1 success and 1 failure to each group—provides near-nominal coverage across scenarios.

Absolute vs. Relative Difference

Absolute difference (what this calculator gives): "Treatment improved outcomes by 16 percentage points"

Relative difference: Relative risk: 78%/62% = 1.26 (26% relative improvement) Odds ratio: Different yet calculation

Both are valid; they answer different questions:

  • Absolute: How many more out of 100?
  • Relative: How many times higher is the rate?

Sample Size Considerations

The margin of error depends on:

  • Both sample sizes (larger = narrower CI)
  • Both proportions (nearer 50% = wider CI)
  • Confidence level

For equal proportions (p₁ ≈ p₂ ≈ p):

ME ≈ z* × √[2p(1-p)/n] when n₁ = n₂ = n

Planning requires specifying expected proportions and desired precision.

Overlap of Individual CIs

A common mistake: concluding proportions are significantly different if their individual CIs don't overlap, or not different if they do overlap.

This is wrong because:

  • Non-overlapping CIs almost always mean significance
  • BUT overlapping CIs don't always mean non-significance

Always use the CI for the difference, not visual inspection of individual CIs.

One-Sided Intervals

Sometimes you only care about one direction:

  • "Is the new treatment at least as good?"
  • "Is the defect rate lower?"

One-sided 95% CI uses z* = 1.645 instead of 1.96, giving a bound in one direction only.

Non-Inferiority Analysis

In clinical trials, you might test whether a new treatment is "not worse by more than Δ":

If the lower bound of CI > -Δ (the non-inferiority margin), the new treatment is non-inferior.

Example: CI = (-2% to 8%), non-inferiority margin = -5% Since -2% > -5%, non-inferiority is established.

Related Calculations

Summary

Confidence intervals for the difference in proportions quantify how success rates differ between groups and the precision of that estimate. Whether comparing treatments, testing website versions, or tracking survey changes over time, the interval reveals both the magnitude of difference and whether it's statistically meaningful. The Agresti-Caffo adjustment improves coverage for small samples. Remember: what matters is whether zero falls inside the interval—if it doesn't, the difference is statistically significant at the corresponding confidence level.

Related Tools