Tukey HSD Calculator

What Is Tukey's HSD Test?

Tukey's HSD (Honestly Significant Difference) is a post-hoc test that identifies which specific group pairs differ after ANOVA finds a significant overall effect.

Question answered: "Given that groups differ somewhere, which specific pairs are different?"

Example: ANOVA shows teaching methods differ significantly. Tukey HSD reveals: Method A ≠ Method C, but Method B doesn't differ from either.

Why Post-Hoc Tests Are Needed

ANOVA only tells you that at least one group differs—not which ones.

Significant F-statistic → "Groups differ somewhere"

To find where, you need pairwise comparisons. But multiple comparisons inflate Type I error.

The Multiple Comparisons Problem

With 4 groups, there are 6 pairwise comparisons.

If each uses α = 0.05:

• P(at least one false positive) = 1 - (0.95)⁶ = 26.5%

With 5 groups (10 comparisons): 40% With 6 groups (15 comparisons): 54%

Tukey's solution: Control family-wise error rate at α across ALL comparisons simultaneously.

How Tukey's HSD Works

The Test Statistic

$q = \frac{M_{\text{larger}} - M_{\text{smaller}}}{SE}$

Where $SE = \sqrt{MSW / n}$

This follows the Studentized Range Distribution with parameters:

• k = number of groups
• df = degrees of freedom for error (from ANOVA)

Critical Value

The critical q-value depends on:

• Number of groups (k)
• Error df
• Desired α (typically 0.05)

If |q| > q_critical, the pair differs significantly.

Minimum Significant Difference

$HSD = q_{\text{critical}} \times \sqrt{\frac{MSW}{n}}$

Any two means differing by more than HSD are significantly different.

Example Calculation

ANOVA results: Four groups, MSW = 25, n = 10 per group, df_error = 36

Group means: A = 72, B = 78, C = 81, D = 85

Step 1: Calculate SE SE = √(25/10) = 1.58

Step 2: Find critical q For k = 4, df = 36, α = 0.05: q_crit ≈ 3.81

Step 3: Calculate HSD HSD = 3.81 × 1.58 = 6.02

Step 4: Compare differences to HSD

Conclusion: D > A, D > B, C > A. Other pairs don't differ significantly.

Confidence Intervals

Tukey provides simultaneous confidence intervals:

$CI = (M_1 - M_2) \pm HSD$

These intervals have 95% simultaneous coverage—95% probability that ALL intervals contain their true differences.

For D vs A: 13 ± 6.02 = (6.98, 19.02)

The interval excludes 0, confirming significance.

Unequal Sample Sizes

Original Tukey assumes equal n. For unequal groups:

Tukey-Kramer Method

$SE = \sqrt{\frac{MSW \times (1/n_1 + 1/n_2)}{2}}$

Calculate separately for each pair.

This is the standard approach in most software.

Comparison with Other Post-Hoc Tests

Bonferroni

Divides α by number of comparisons.

Advantages: Works for any comparisons, including planned contrasts. Disadvantages: More conservative than Tukey for many comparisons.

Scheffe

Most conservative. Allows ANY linear contrast, not just pairwise.

Use when: You want to test complex comparisons (e.g., Group A+B vs C+D).

Dunnett

Compares each group to a single control.

Use when: You have one reference group (e.g., placebo) and only care about comparisons to it.

Fisher's LSD

No correction—just multiple t-tests.

Problem: Doesn't control family-wise error. Generally not recommended.

Games-Howell

Doesn't assume equal variances.

Use when: Variances are unequal.

When to Use Tukey HSD

Appropriate:

• All pairwise comparisons are of interest
• Sample sizes are equal or similar
• ANOVA assumptions (especially equal variances) are met
• You want good power with family-wise error control

Not appropriate:

• Only specific comparisons planned (use Bonferroni or planned contrasts)
• Comparing to control only (use Dunnett)
• Variances clearly unequal (use Games-Howell)
• ANOVA was not significant (no post-hoc needed)

Assumptions

Equal Variances

Tukey assumes homogeneous variances (same as ANOVA).

If violated: Use Games-Howell or Tamhane's T2.

Normality

Populations approximately normal, or adequate sample sizes.

Independence

Observations are independent within and between groups.

Significant ANOVA

Only conduct post-hoc tests if overall F is significant.

Exception: Some pre-planned comparisons can proceed regardless.

Applications

Clinical Research

After ANOVA shows drug doses differ: "Which doses are different from placebo?"

Marketing

After ANOVA shows ad types differ: "Which ads outperform others?"

Education

After ANOVA shows curricula differ: "Which curriculum is best?"

Agriculture

After ANOVA shows fertilizers differ: "Which fertilizers produce higher yields?"

Visualizing Results

Compact Letter Display

Groups sharing a letter don't differ significantly:

D and C share 'A'—not significantly different. A and B share 'C'—not significantly different. D and A share nothing—significantly different.

Common Mistakes

Conducting Without Significant ANOVA

Post-hoc tests are for exploring significant overall effects.

Reporting Only Significant Pairs

Report all comparisons for transparency.

Ignoring Effect Sizes

Statistical significance doesn't imply practical importance.

Wrong Test for Unequal Variances

Use Games-Howell, not Tukey, when variances differ.

Reporting Results

APA format:

"Post-hoc Tukey HSD tests revealed that Group D (M = 85) scored significantly higher than Group A (M = 72, p < .001, 95% CI [6.98, 19.02]) and Group B (M = 78, p = .02, 95% CI [0.98, 13.02]). Group C (M = 81) also scored significantly higher than Group A (p = .01, 95% CI [2.98, 15.02]). No other pairwise differences were significant."

Historical Context

John Tukey developed the Honestly Significant Difference test in 1949 as part of his broader contributions to statistical methodology.

The name "Honestly Significant Difference" reflected Tukey's concern with the dishonesty of uncontrolled multiple comparisons—cherry-picking significant results from many tests.

Tukey's work emphasized the importance of controlling error rates across multiple inferences, a principle that remains central to modern statistics.

Related Calculations

• One-Way ANOVA — Prerequisite test
• Two-Way ANOVA — Post-hocs for each factor
• Two-Sample T-Test — Single comparison
• Cohen's d Calculator — Effect sizes

Summary

Tukey's HSD test identifies which group pairs differ significantly after ANOVA detects an overall effect. It controls family-wise error rate at α across all pairwise comparisons by using the Studentized Range Distribution. Calculate the HSD threshold, then compare each pairwise mean difference to this value. Groups differing by more than HSD are significantly different. The Tukey-Kramer modification handles unequal sample sizes. Report all comparisons, confidence intervals, and p-values for complete and transparent results.