Bonferroni Correction Calculator

What Is the Bonferroni Correction?

The Bonferroni correction adjusts significance levels when performing multiple statistical tests. It controls the family-wise error rate (FWER)—the probability of making at least one Type I error across all tests.

The problem: With 20 tests at α = 0.05, expect 1 false positive even if no true effects exist.

The solution: Divide α by the number of tests.

$\alpha_{adjusted} = \frac{\alpha}{m}$

Where m is the number of tests.

The Multiple Comparisons Problem

Single Test

With α = 0.05, there's a 5% chance of false positive.

Multiple Tests

Formula: $1 - (1 - \alpha)^m$

Without correction, false positives accumulate rapidly.

How Bonferroni Works

Adjusted Alpha Level

If testing at overall α = 0.05 with m = 10 comparisons:

$\alpha_{Bonferroni} = \frac{0.05}{10} = 0.005$

Each individual test uses α = 0.005.

Adjusted P-Values

Alternatively, multiply each p-value by m:

$p_{adjusted} = \min(p \times m, 1)$

Compare adjusted p-values to original α = 0.05.

Example

Scenario: Compare 4 treatment groups pairwise (6 comparisons).

Adjusted α = 0.05/6 = 0.0083

Only comparisons with raw p < 0.0083 remain significant.

When to Use Bonferroni

Appropriate

• Pre-planned comparisons (not data-driven)
• Confirmatory analysis
• When false positives are costly
• Small number of tests (< 20)
• Tests are independent or positively correlated

Less Appropriate

• Many tests (> 20): Too conservative
• Exploratory analysis: Use less strict methods
• Dependent tests: Other methods may be more powerful
• When false negatives are costly

Advantages

Simplicity

Easy to calculate and explain.

Conservative

Guarantees FWER ≤ α.

No Assumptions

Works for any combination of tests, regardless of dependence.

Flexibility

Can be applied to any set of p-values.

Disadvantages

Overly Conservative

Especially with many tests, real effects may be missed.

Example: With 100 tests, α_adjusted = 0.0005. Even moderate true effects may not achieve significance.

Reduced Power

As the number of tests increases, power to detect true effects decreases.

Same Adjustment for All

Doesn't distinguish between tests of varying importance.

Alternatives to Bonferroni

Holm-Bonferroni (Step-Down)

More powerful than Bonferroni while still controlling FWER.

1. Order p-values from smallest to largest
2. Compare p_(i) to α/(m - i + 1)
3. Reject until first non-rejection, then stop

Holm rejects more (3 vs 2) while maintaining FWER control.

Šidák Correction

Slightly less conservative than Bonferroni:

$\alpha_{Šidák} = 1 - (1 - \alpha)^{1/m}$

For m = 10: α_Šidák = 0.00512 vs Bonferroni's 0.00500.

Benjamini-Hochberg (FDR)

Controls false discovery rate rather than FWER. More powerful for large-scale testing.

Tukey HSD

For all pairwise comparisons after ANOVA. Uses studentized range distribution.

Dunnett

For comparing each treatment to a single control.

Comparison of Methods

How Many Tests Are Being Done?

Counting tests correctly is crucial:

Count All Tests Performed

Including those that "didn't work out."

Include Preliminary Tests

If you tested normality, homogeneity, etc., those count.

Don't Count:

• Descriptive statistics
• Confidence intervals for the same parameter
• Subgroup analyses declared a priori

Applications

Post-Hoc Comparisons

After ANOVA, comparing all pairs of group means.

Subgroup Analyses

Testing treatment effects in multiple patient subgroups.

Multiple Endpoints

Clinical trials with several outcome measures.

Genetic Studies

Though usually FDR methods are preferred for thousands of tests.

Imaging Studies

Multiple brain regions or voxels.

Reporting Results

Method 1: Report adjusted alpha

"Bonferroni correction was applied for 6 pairwise comparisons, using α = 0.05/6 = 0.0083. Two comparisons remained significant: A vs B (p = .003) and B vs C (p = .008)."

Method 2: Report adjusted p-values

"P-values were adjusted using Bonferroni correction. Significant differences were found between A and B (p_adj = .018) and between B and C (p_adj = .048)."

Common Mistakes

Correcting When Not Needed

A single pre-specified primary comparison doesn't need correction.

Under-Counting Tests

Forgetting preliminary analyses or abandoned tests.

Over-Correcting

Correcting for tests in other studies or unrelated hypotheses.

Using with Thousands of Tests

Bonferroni is too conservative for genomics-scale testing. Use FDR methods.

Historical Context

Carlo Emilio Bonferroni (1892-1960) was an Italian mathematician who developed probability inequalities used in this correction.

The Bonferroni inequality states: $P(A_1 \cup A_2 \cup \cdots \cup A_m) \leq \sum P(A_i)$

This inequality guarantees that dividing α by m controls the overall error rate.

Related Calculations

• Tukey HSD Calculator — Post-hoc for ANOVA
• One-Way ANOVA — Omnibus test before comparisons
• Two-Sample T-Test — Individual comparisons
• Chi-Square Test — Categorical comparisons

Summary

The Bonferroni correction controls family-wise error rate by dividing α by the number of tests or multiplying p-values by the number of tests. It's simple and robust but conservative—consider Holm-Bonferroni for more power while maintaining FWER control, or FDR methods for many tests. Count all tests performed, including preliminary analyses. Report either adjusted alpha level or adjusted p-values. Use it for confirmatory analyses with a small number of pre-planned comparisons.