Propensity Score Calculator

Use propensity scores to reduce confounding in observational studies when randomization isn't possible. The propensity score balances treatment and control groups on observed covariates, mimicking a randomized experiment.

What Is a Propensity Score?

The propensity score is the probability of receiving treatment given observed covariates:

$e(X) = P(T = 1 | X)$

Where:

• T = treatment indicator (1 = treated, 0 = control)
• X = vector of observed covariates

Why Propensity Scores Work

Key theorem (Rosenbaum & Rubin, 1983): If treatment assignment is ignorable given X, it's also ignorable given e(X).

This means instead of matching on many covariates, you can match on a single score.

Estimating Propensity Scores

Logistic Regression (Most Common)

$\log\left(\frac{e(X)}{1-e(X)}\right) = \beta_0 + \beta_1 X_1 + ... + \beta_k X_k$

Fit logistic regression with treatment as outcome, covariates as predictors.

Alternative Methods

Machine learning methods may improve prediction but reduce interpretability.

Using Propensity Scores

1. Matching

Pair treated units with control units having similar propensity scores.

Nearest neighbor matching: For each treated unit, find the control with closest propensity score.

Caliper matching: Match only if |e_treated - e_control| < caliper (often 0.2 × SD of propensity score).

1:k matching: Match each treated unit to k controls (increases precision, may reduce balance).

2. Stratification (Subclassification)

Divide sample into strata based on propensity score quintiles:

$\hat{ATE} = \sum_{s=1}^{5} \frac{n_s}{N} (\bar{Y}_{1s} - \bar{Y}_{0s})$

Five strata typically remove 90% of bias from measured confounders.

3. Inverse Probability Weighting (IPW)

Weight each observation by inverse of propensity score:

For treated: $w = \frac{1}{e(X)}$

For control: $w = \frac{1}{1-e(X)}$

IPW estimator for ATE: $\hat{ATE}_{IPW} = \frac{1}{n}\sum_{i=1}^{n}\left[\frac{T_i Y_i}{e(X_i)} - \frac{(1-T_i)Y_i}{1-e(X_i)}\right]$

4. Doubly Robust Estimation

Combines outcome modeling with propensity weighting:

$\hat{ATE}_{DR} = \frac{1}{n}\sum_{i=1}^{n}\left[\hat{m}_1(X_i) - \hat{m}_0(X_i) + \frac{T_i(Y_i - \hat{m}_1(X_i))}{e(X_i)} - \frac{(1-T_i)(Y_i - \hat{m}_0(X_i))}{1-e(X_i)}\right]$

Consistent if either the propensity or outcome model is correct.

Checking Balance

After applying propensity scores, verify that covariates are balanced between groups.

Standardized Mean Difference (SMD)

$SMD = \frac{\bar{X}_{treated} - \bar{X}_{control}}{\sqrt{(s^2_{treated} + s^2_{control})/2}}$

Target: |SMD| < 0.1 for all covariates

Variance Ratio

$VR = \frac{s^2_{treated}}{s^2_{control}}$

Target: 0.5 < VR < 2.0

Visual Checks

• Love plots (SMD before vs. after)
• Propensity score distributions by group
• Covariate distributions within strata

Example Analysis

Research question: Effect of job training on earnings

Data:

• 185 treated (received training)
• 429 control (no training)
• Covariates: age, education, prior earnings, race, married

Step 1: Estimate propensity scores

Logistic regression:
- Treatment ~ age + education + earnings_1974 + race + married
- Predicted probabilities = propensity scores

Step 2: Check overlap

• Treated: mean e(X) = 0.42, range [0.05, 0.89]
• Control: mean e(X) = 0.31, range [0.02, 0.78]
• Good overlap in middle, some extreme values

Step 3: Match or weight

• 1:1 nearest neighbor matching with caliper = 0.05
• 150 matched pairs (35 treated dropped—no good match)

Step 4: Check balance

• Before matching: age SMD = 0.35, education SMD = 0.28
• After matching: age SMD = 0.04, education SMD = 0.03

Step 5: Estimate effect

• ATT = $1,794 (95$487, $3,101)
• Interpretation: Training increased earnings by ~$1,800

Treatment Effect Estimands

Different methods estimate different quantities. Be clear about your target.

Common Problems

Lack of Overlap

If propensity distributions don't overlap, comparison is extrapolation.

Solutions:

• Trim extreme propensity scores (e.g., drop e < 0.1 or e > 0.9)
• Report effects only for overlap region
• Consider different estimand

Extreme Weights

With IPW, small propensity scores → huge weights → unstable estimates.

Solutions:

• Trim or stabilize weights
• Stabilized weights: multiply by marginal treatment probability
• Set maximum weight (e.g., 10× mean)

Model Misspecification

If propensity model is wrong, bias remains.

Solutions:

• Include interactions and polynomials
• Use doubly robust methods
• Try multiple specifications

Sensitivity Analysis

Propensity scores only control for observed confounders. Unobserved confounding can bias results.

Sensitivity analysis asks: How strong would unmeasured confounding need to be to explain away the effect?

Common approaches:

• Rosenbaum bounds
• E-values
• Sensitivity functions

Propensity Score vs. Alternatives

Reporting Guidelines

Include:

• How propensity scores were estimated (model, variables)
• Overlap assessment
• Balance statistics (before/after)
• Number of observations used
• Which estimand (ATE, ATT)
• Sensitivity analysis for unmeasured confounding

Example: "Propensity scores were estimated via logistic regression including age, sex, baseline severity, and comorbidities. After 1:1 nearest-neighbor matching (caliper = 0.2 SD), all standardized mean differences were < 0.1. The ATT was 5.2 points (95% CI: 2.1, 8.3). Sensitivity analysis indicated that an unmeasured confounder would need OR > 2.5 with both treatment and outcome to nullify the effect."

Historical Context

Paul Rosenbaum and Donald Rubin introduced propensity scores in 1983, building on Rubin's causal inference framework. The method transformed observational research by providing a practical way to address confounding. Extensions like doubly robust estimation and machine learning-based scores continue to advance the field.

Related Calculations

• Logistic Regression Calculator — Estimate propensity scores
• Odds Ratio Calculator — Effect measures
• Relative Risk Calculator — Risk comparison
• Confidence Interval Calculator — Inference

Summary

Propensity scores estimate the probability of treatment given covariates, enabling causal inference in observational studies. Use matching, stratification, or weighting to balance groups. Check balance with standardized differences; target |SMD| < 0.1. Be aware of limitations: propensity scores only control for observed confounders. Always conduct sensitivity analysis for unmeasured confounding. Report your methods, balance checks, and the specific estimand (ATE vs ATT).