Logistic Regression Calculator

What Is Logistic Regression?

Logistic regression models the probability of a binary outcome (yes/no, success/failure, 1/0) based on one or more predictor variables. Instead of predicting a continuous value, it predicts the probability of belonging to a category.

Question answered: "What factors affect the probability of this outcome occurring?"

Examples:

• "What factors predict whether a customer will churn?"
• "How do patient characteristics affect probability of disease?"
• "What predicts whether a loan applicant will default?"

Why Not Linear Regression?

Problem with linear regression for binary outcomes:

Linear regression can predict values below 0 or above 1—impossible for probabilities.

Solution: Logistic regression uses the logistic function to constrain predictions to (0, 1).

The Logistic Function

$P(Y = 1) = \frac{1}{1 + e^{-z}}$

Where $z = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \ldots$

This S-shaped curve maps any real number to a probability between 0 and 1.

The Logit Transformation

Rearranging:

$\log\left(\frac{P}{1-P}\right) = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \ldots$

The left side is the log odds (logit) of the outcome.

The right side is linear in the predictors.

Interpreting Coefficients

Coefficient interpretation differs from linear regression:

$\beta_1$ = change in log odds per unit increase in $X_1$

More intuitively, convert to odds ratio:

$OR = e^{\beta}$

Example: If β₁ = 0.7 for a predictor:

• OR = e^0.7 = 2.01
• Each unit increase in X₁ doubles the odds of the outcome

Example: Customer Churn

Data: 1,000 customers with churn status (1 = churned, 0 = retained)

Predictors: Monthly charges, contract length, tenure (months)

Output:

Interpretations:

• Each $1 increase in monthly charges: 3% increase in churn odds
• Two-year contract vs. monthly: 84% reduction in churn odds
• Each additional month of tenure: 4% reduction in churn odds

Odds Ratios (OR)

Confidence interval for OR: If CI includes 1, the effect is not statistically significant.

Prediction

Predicted Probability

1. Calculate z = β₀ + β₁X₁ + β₂X₂ + ...
2. Apply logistic function: P = 1 / (1 + e^(-z))

Example: For a customer with:

• Monthly charges = $80
• Two-year contract
• Tenure = 24 months

z = -1.24 + 0.031(80) - 1.85(1) - 0.045(24) = -1.69 P(churn) = 1 / (1 + e^1.69) = 0.16 (16% probability)

Classification

Apply a threshold (typically 0.5):

• If P ≥ 0.5, predict Y = 1
• If P < 0.5, predict Y = 0

Threshold can be adjusted based on costs of false positives vs. false negatives.

Model Assessment

Likelihood Ratio Test (Overall Model)

Compares fitted model to null model (intercept only).

H₀: All β = 0 H₁: At least one β ≠ 0

Chi-square statistic with df = number of predictors.

Pseudo R² Measures

Unlike linear regression, logistic regression has no true R².

McFadden's R²: $1 - (\log L_{\text{full}} / \log L_{\text{null}})$

Nagelkerke R²: Adjusted version that can reach 1

Interpretation differs from linear R²—values above 0.2 are often considered acceptable.

Hosmer-Lemeshow Test

Tests whether observed and predicted probabilities match.

Non-significant p-value suggests good fit.

Classification Metrics

Use ROC-AUC for threshold-independent evaluation.

Assumptions

Binary Outcome

Dependent variable is binary (or can be treated as such).

Independence

Observations are independent.

No Perfect Multicollinearity

Predictors are not perfectly correlated.

Linearity in the Logit

The relationship between predictors and log odds is linear.

Check: Component + residual plots

Large Sample Size

General guideline: at least 10-20 events per predictor.

With rare outcomes, you need larger samples.

Categorical Predictors

Use dummy variables, same as multiple regression:

Three-category variable → two dummy variables

Reference category is coded 0 on all dummies.

Interaction Effects

X₁ × X₂ interaction:

log(P/(1-P)) = β₀ + β₁X₁ + β₂X₂ + β₃(X₁ × X₂)

The effect of X₁ depends on the value of X₂.

Maximum Likelihood Estimation

Unlike linear regression (least squares), logistic regression uses maximum likelihood:

Find coefficients that maximize the probability of observing the actual data.

No closed-form solution—iterative algorithms (Newton-Raphson) find the optimum.

Common Issues

Complete Separation

When a predictor perfectly predicts the outcome:

• All cases with X > threshold have Y = 1
• Coefficient goes to infinity

Solutions: Firth's penalized likelihood, exact logistic regression

Imbalanced Classes

When one outcome is rare (e.g., 1% fraud):

• Model may predict majority class for everything
• Accuracy is misleading

Solutions: Class weights, oversampling minority class, different threshold

Sample Size

With many predictors or rare outcomes, estimation becomes unstable.

Rule of thumb: 10+ events per predictor variable.

Extensions

Multinomial Logistic Regression

Outcome has 3+ unordered categories.

Example: Political party affiliation (Democrat, Republican, Independent)

Ordinal Logistic Regression

Outcome has ordered categories.

Example: Disease severity (mild, moderate, severe)

Conditional Logistic Regression

For matched case-control studies.

Reporting Results

Standard format:

"Logistic regression was performed to examine factors associated with customer churn. The model was statistically significant, χ²(3) = 245.8, p < .001, Nagelkerke R² = .31.

Monthly charges (OR = 1.03, 95% CI [1.02, 1.04], p < .001), two-year contract (OR = 0.16, 95% CI [0.10, 0.25], p < .001), and tenure (OR = 0.96, 95% CI [0.94, 0.97], p < .001) were significant predictors."

Historical Context

David Cox formalized logistic regression in 1958, though the logistic function had been studied earlier.

The method became practical with computing power in the 1960s-70s and is now fundamental in epidemiology, marketing, machine learning, and any field predicting binary outcomes.

The odds ratio interpretation connects logistic regression to case-control study design common in epidemiology.

Related Calculations

• ROC-AUC Calculator — Model evaluation
• Odds Ratio Calculator — Effect measure
• Multiple Regression — Continuous outcomes
• Chi-Square Test — Categorical associations

Summary

Logistic regression models the probability of a binary outcome using the logistic function, ensuring predictions stay between 0 and 1. Coefficients represent changes in log odds; exponentiate to get odds ratios for interpretation. Assess overall model with likelihood ratio test, evaluate classification with confusion matrix metrics and ROC-AUC. Assumptions include independence, linearity in logit, and adequate sample size (10+ events per predictor). Report odds ratios with confidence intervals and significance tests for practical interpretation.