Sign Test Calculator

Use the sign test when you have paired observations and simply want to know if one measurement tends to be higher than the other. It's the go-to choice when your data has outliers, isn't normally distributed, or when you only care about direction (did scores go up or down?) rather than by how much. This makes it ideal for quick before-after comparisons or preference studies.

What Is the Sign Test?

The sign test is the simplest non-parametric test for paired data. It uses only the direction (sign) of differences between pairs, ignoring their magnitude.

Question answered: "Is there a consistent tendency for one condition to be higher than another?"

Examples:

• Do patients tend to improve after treatment? (before-after)
• Do people prefer Product A over Product B? (paired comparison)
• Is there a consistent difference between two measurement methods?

How It Works

Step 1: Calculate Differences

For each pair, compute the difference (After - Before).

Step 2: Count Signs

• Positive signs: 1
• Negative signs: 4
• Zeros: 1 (excluded)

n = 5 (non-zero differences)

Step 3: Apply Binomial Test

Under null hypothesis (no difference), positive and negative signs are equally likely.

Test statistic: S = min(positive, negative) = 1

P-value: $P(X \leq 1)$ where $X \sim \text{Binomial}(5, 0.5)$

$P = 2 \times \sum_{k=0}^{1} \binom{5}{k}(0.5)^5 = 2 \times (0.031 + 0.156) = 0.375$

Conclusion: No significant difference ($p = 0.375$).

The Test Statistic

For a two-tailed test:

$S = \min(n_+, n_-)$

Where $n_+$ is the count of positive differences and $n_-$ is the count of negative differences.

Exact P-Value (Small Samples)

$p = 2 \times P(X \leq S) = 2 \times \sum_{k=0}^{S} \binom{n}{k}(0.5)^n$

Normal Approximation (Large Samples)

For n > 25:

$z = \frac{S - 0.5n}{\sqrt{0.25n}}$

With continuity correction:

$z = \frac{|S - 0.5n| - 0.5}{\sqrt{0.25n}}$

Handling Ties (Zero Differences)

Zero differences provide no information about direction.

Standard approach: Exclude from analysis.

Alternative: Assign half to each sign (controversial).

Comparison with Wilcoxon Signed-Rank Test

Rule of thumb: Use Wilcoxon signed-rank when differences can be meaningfully ranked. Use sign test when only direction matters or when there are many ties in magnitudes.

One-Sample Sign Test

Test whether the median equals a hypothesized value:

1. Calculate differences from hypothesized median
2. Count positive and negative signs
3. Apply binomial test

Example: Test if median income = $50,000

Positive: 3, Negative: 2, p = 1.00 (not significant)

Confidence Interval for Median

The sign test can construct a confidence interval for the population median difference.

For n pairs, order the differences and find the appropriate quantiles based on binomial distribution.

Applications

Medical Research

• Simple before-after comparisons
• Crossover trials with ordinal outcomes
• Preference studies

Psychology

• Paired preference tests
• Treatment effectiveness (better/worse/same)

Quality Control

• Comparing two measurement methods
• Before-after process changes

Market Research

• Product preference (A vs B)
• Rating comparisons

Advantages

Minimal Assumptions

• No normality requirement
• No requirement for continuous data
• Works with ordinal data
• Only requires paired observations

Robustness

• Unaffected by outliers
• Works even when magnitude is unmeasurable
• Valid for any distribution shape

Simplicity

• Easy to understand and calculate
• Results are intuitive (count of improvements)

Disadvantages

Low Power

By ignoring magnitude, the sign test discards valuable information.

Example: Two studies both show 15 improvements out of 20 patients.

• Study A: Improvements average +2 points
• Study B: Improvements average +20 points

The sign test gives identical results, but Study B shows much stronger effect.

Large Sample Requirement

To achieve adequate power, sign test needs larger samples than parametric alternatives.

When to Choose the Sign Test

Use Sign Test When:

• Data is ordinal (only direction meaningful)
• Many ties in difference magnitudes
• Distribution is highly asymmetric
• Quick, simple analysis needed
• Extreme robustness required

Use Wilcoxon Signed-Rank When:

• Differences can be meaningfully ranked
• You want more power
• Distribution of differences is roughly symmetric

Use Paired T-Test When:

• Differences are approximately normal
• Maximum power is needed
• Interval/ratio scale data

Example: Complete Analysis

Scenario: 12 patients rate pain (1-10) before and after treatment.

Results:

• Positive: 1
• Negative: 10
• Ties: 1 (excluded)
• n = 11

P-value: $P(X \leq 1) = 0.012$ (two-tailed)

Conclusion: Significant reduction in pain after treatment ($p = 0.012$).

Reporting Results

APA format:

"A sign test indicated that pain scores were significantly lower after treatment, with 10 of 11 patients showing improvement (p = .012)."

Or more formally:

"A sign test revealed a significant treatment effect, S = 1, n = 11, p = .012, with the majority of patients (91%) showing reduced pain scores."

Historical Context

The sign test is one of the oldest statistical procedures, dating back to John Arbuthnot's 1710 study of birth sex ratios.

Its simplicity made it practical before modern computing. While more powerful tests are now available, the sign test remains valuable for its minimal assumptions and ease of interpretation.

Related Calculations

• Wilcoxon Signed-Rank Test — More powerful paired test
• Paired T-Test — Parametric paired test
• Binomial Test — Underlying distribution
• McNemar's Test — For paired categorical data

Summary

The sign test compares paired observations using only the direction of differences. Count positive and negative differences, exclude ties, and apply a binomial test with p = 0.5. It's the simplest non-parametric paired test with minimal assumptions but lower power than Wilcoxon signed-rank. Use it when data is ordinal, differences have many ties, or extreme robustness is needed. Report the count of each sign direction, sample size, and p-value.