Fisher's Exact Test Calculator

What Is Fisher's Exact Test?

Fisher's Exact Test determines whether there's an association between two categorical variables when sample sizes are small. Unlike chi-square, it calculates exact probabilities rather than relying on approximations.

Question answered: "Is there a relationship between these two categorical variables?" (when sample size is too small for chi-square)

Examples:

• "Is a rare disease associated with a rare exposure?" (few cases in each cell)
• "Did treatment improve outcome in a small clinical trial?"
• "Is there gender bias in a small hiring dataset?"

Why "Exact"?

Chi-square uses a continuous distribution to approximate discrete data. This approximation fails with small samples.

Fisher's Exact Test:

• Calculates exact probability of observed (or more extreme) tables
• Valid for any sample size
• No minimum expected frequency requirement

The 2×2 Table

Standard layout:

Given fixed marginal totals (row and column totals), the probability of any specific table is calculated using the hypergeometric distribution.

The Calculation

P(observed table) = [(a+b)!(c+d)!(a+c)!(b+d)!] / [n!a!b!c!d!]

For a two-tailed test, sum probabilities of:

• The observed table
• All tables more extreme (with same marginals)

Example

Data: Testing new treatment on 15 patients

Question: Is improvement associated with treatment?

Probability calculation: P = (10!)(5!)(11!)(4!) / (15!)(8!)(2!)(3!)(2!) P = 0.286 (for this exact table)

Two-tailed p-value: Sum this with more extreme tables = 0.608

Conclusion: No significant association (p > 0.05).

One-Tailed vs. Two-Tailed

Two-Tailed (Default)

Tests for any association, regardless of direction.

Use when: You don't have a prior hypothesis about direction.

One-Tailed

Tests for association in a specific direction.

Use when: You only care if treatment improves (not worsens) outcome.

One-tailed p-value ≈ two-tailed / 2 (approximately).

When to Use Fisher's Exact Test

Traditional rule: When any expected frequency < 5.

Modern recommendation: Always valid; chi-square is an approximation that's usually unnecessary with modern computing.

Practical guidance:

• Small total sample (n < 20)
• Any cell with count < 5
• Rare events or small expected frequencies
• When exact p-value is important (regulatory submissions)

Comparison with Chi-Square

For large samples: Both give similar results; chi-square is computationally simpler.

For small samples: Fisher's is more accurate.

Extensions Beyond 2×2

R×C Tables

Fisher's exact test extends to larger tables but becomes computationally intensive.

Most software handles tables up to moderate size.

For very large tables, chi-square approximation is usually adequate anyway.

Mid-P Adjustment

Some statisticians recommend a "mid-P" version that counts half the probability of the observed table.

Rationale: Reduces conservatism of exact test. Controversy: Purists prefer the fully exact version.

Assumptions

Independence

Observations are independent—sampling one subject doesn't affect another.

Fixed Marginals

Row and column totals are considered fixed by the sampling design.

In practice: The test is valid even if marginals weren't literally fixed.

Random Sampling

Sample represents the population of interest.

Applications

Rare Disease Epidemiology

Studying rare exposures and rare outcomes.

Case-control studies with few cases.

Small Clinical Trials

Pilot studies, rare disease trials, early phase research.

Genetics

Testing associations with rare alleles.

Quality Control

Comparing defect rates when defects are rare.

Behavioral Research

Small sample experiments, case studies.

Measures of Association

Fisher's test gives p-value. For effect size, calculate:

Odds Ratio

OR = (a × d) / (b × c)

From the example: OR = (8 × 2) / (2 × 3) = 2.67

Treatment patients had 2.67 times the odds of improvement.

Relative Risk

RR = [a/(a+b)] / [c/(c+d)]

RR = (8/10) / (3/5) = 0.80 / 0.60 = 1.33

Treatment patients were 1.33 times as likely to improve.

Confidence Intervals

Fisher's test can provide exact confidence intervals for the odds ratio.

These are often asymmetric and more accurate than normal-approximation intervals for small samples.

Common Misconceptions

"Use Chi-Square First, Then Fisher If Needed"

Modern practice: Fisher's is always valid. Use it when exact p-values matter or samples are small.

"Fisher's Is Conservative"

Slightly true. The discrete nature means p-values can't always be exactly 0.05. Mid-P adjustment addresses this.

"Fisher's Only for 2×2"

Extensions exist for larger tables, though computation increases.

Reporting Results

APA format:

"Fisher's exact test revealed no significant association between treatment and improvement, p = .61. The odds ratio was 2.67, 95% CI [0.28, 37.8], indicating a non-significant trend toward better outcomes with treatment."

Include:

• Exact p-value
• Effect measure (OR or RR)
• Confidence interval
• Descriptive frequencies or proportions

Historical Context

Ronald Fisher developed this test in the 1930s, reportedly for a famous "lady tasting tea" experiment.

A colleague claimed she could detect whether tea or milk was poured first into a cup. Fisher designed an experiment with 8 cups and calculated exact probabilities to test her claim.

This anecdote (whether fully accurate or not) illustrates the test's purpose: exact inference with small samples where approximations fail.

The test exemplifies Fisher's broader contribution of developing exact statistical methods alongside the large-sample approximations that dominate textbooks.

Computational Considerations

For 2×2 tables: Trivial computation.

For larger tables: The test requires summing over all possible tables with the same marginals. This can be millions of tables.

Network algorithm: Developed in the 1980s-90s, makes larger tables feasible.

Modern software: Handles moderate-sized tables routinely.

Related Calculations

• Chi-Square Independence Test — Large samples
• Odds Ratio Calculator — Effect size for 2×2
• Relative Risk Calculator — Cohort effect size
• Two-Proportion Z-Test — Large sample comparison
• McNemar's Test — Paired categorical data

Summary

Fisher's Exact Test determines whether two categorical variables are associated by calculating exact probabilities based on the hypergeometric distribution. Unlike chi-square, it requires no minimum sample size or expected frequency. The test considers all possible tables with the same row and column totals and sums probabilities of tables as extreme or more extreme than observed. Use Fisher's for small samples, rare events, or when exact p-values are required. Report the exact p-value, odds ratio or relative risk, and confidence intervals for complete interpretation.