Linear Regression Calculator

What Is Simple Linear Regression?

Simple linear regression models the relationship between two variables by fitting a straight line to the data. It allows you to predict Y (the outcome) from X (the predictor).

Question answered: "What's the best-fitting line, and how well does X predict Y?"

Examples:

• "How do sales change with advertising spending?"
• "Predict exam score from study hours."
• "What's the relationship between temperature and ice cream sales?"

The Regression Equation

$Y = a + bX$

Or in alternative notation:

$\hat{Y} = \beta_0 + \beta_1 X$

Where:

• $a$ (or $\beta_0$) = Y-intercept (value of Y when X = 0)
• $b$ (or $\beta_1$) = slope (change in Y for one-unit increase in X)
• $\hat{Y}$ = predicted Y value

Calculating the Coefficients

Slope:

$b = \frac{\sum_{i=1}^{n}(x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^{n}(x_i - \bar{x})^2}$

Or: $b = S_{xy} / S_{xx} = r \times (s_y / s_x)$

Intercept:

$a = \bar{y} - b \times \bar{x}$

The line always passes through the point $(\bar{x}, \bar{y})$.

Example Calculation

Data: Advertising spend (X, $1000s) and sales (Y, units)

Summary statistics:

• x̄ = 6, ȳ = 288
• Sxx = 40, Sxy = 1100

Slope: b = 1100 / 40 = 27.5

Intercept: a = 288 - 27.5 × 6 = 123

Equation: Ŷ = 123 + 27.5X

Interpretation: Each $1,000 increase in advertising is associated with 27.5 additional unit sales.

Interpreting the Slope

b = 27.5 means:

• For every 1-unit increase in X, Y increases by 27.5 on average
• The relationship is positive (Y increases as X increases)
• This is an average effect—individual observations vary

Units matter: If X is in thousands of dollars and Y is in units sold, then: "$1,000 more advertising → 27.5 more units sold"

Interpreting the Intercept

a = 123 means:

• When X = 0, the predicted Y is 123
• Zero advertising predicts 123 unit sales

Caution: Interpret literally only if X = 0 is meaningful and within your data range.

If your data spans X = 2 to 10, the intercept is extrapolation and may not be meaningful.

Residuals and Fit

Residual = Observed - Predicted

$e_i = y_i - \hat{y}_i$

Residuals should be small and randomly scattered.

R-Squared (Coefficient of Determination)

$R^2 = \text{proportion of variance in Y explained by X}$

$R^2 = 1 - \frac{SS_{residual}}{SS_{total}}$

Or: $R^2 = r^2$ (squared correlation)

For our example: r = 0.995, so R² = 0.99

Interpretation: 99% of variation in sales is explained by advertising spend.

Standard Error of the Estimate

$SE = \sqrt{\frac{\sum_{i=1}^{n}(y_i - \hat{y}_i)^2}{n-2}}$

Measures typical prediction error.

For our example: SE ≈ 7.9 units

Predictions are typically off by about 8 units.

Testing the Slope

Is the slope significantly different from zero?

H₀: β₁ = 0 (no relationship) H₁: β₁ ≠ 0 (relationship exists)

Test statistic:

$t = \frac{b}{SE_b}$

Where $SE_b = SE / \sqrt{S_{xx}}$

If p < α, the slope is statistically significant.

Confidence Interval for Slope

$CI = b \pm t^* \times SE_b$

Gives range of plausible population slopes.

Prediction

Point Prediction

For X = 7: Ŷ = 123 + 27.5(7) = 315.5 units

Prediction Interval

Accounts for uncertainty in both the line and individual variation:

$PI = \hat{Y} \pm t^* \times SE \times \sqrt{1 + \frac{1}{n} + \frac{(X - \bar{x})^2}{S_{xx}}}$

Prediction intervals are always wider than confidence intervals.

Assumptions (LINE)

Linearity

The relationship between X and Y is linear.

Check: Scatterplot, residual plot

Independence

Observations are independent.

Violation: Time series data, clustered data

Normality

Residuals are normally distributed.

Check: Histogram of residuals, QQ plot

Equal Variance (Homoscedasticity)

Residual spread is constant across X values.

Check: Residual plot (should show random scatter)

Residual Diagnostics

Good residual plot:

• Random scatter around zero
• No patterns or trends
• Constant spread

Problems to detect:

• Curvature → non-linear relationship
• Funnel shape → heteroscedasticity
• Clusters → potential subgroups

Common Issues

Extrapolation

Predicting outside the range of observed X values.

If data spans X = 2 to 10, predicting at X = 20 is risky.

Outliers

Unusual points can heavily influence the line.

High leverage: Extreme X values High influence: Points that change the line substantially

Confounding

Third variables may explain the relationship.

Regression doesn't prove causation.

Applications

Business

Sales forecasting, demand modeling, pricing analysis.

Science

Calibration curves, dose-response relationships.

Economics

Consumption functions, price elasticity.

Medicine

Growth charts, treatment response modeling.

Engineering

Quality control, process optimization.

Regression vs. Correlation

Connection: r² = R² in simple regression

Reporting Results

Standard format:

"Simple linear regression revealed that advertising spending significantly predicted sales, b = 27.5, SE = 1.38, t(3) = 19.9, p < .001. The model explained 99% of the variance in sales (R² = .99)."

With equation:

"The regression equation was: Sales = 123 + 27.5 × Advertising, F(1, 3) = 397, p < .001, R² = .99."

Historical Context

Francis Galton pioneered regression in the 1880s studying heredity—specifically, why children of tall parents "regressed" toward average height.

Karl Pearson formalized the mathematics, connecting regression to correlation.

The term "regression" persists even though modern applications rarely involve "regression toward the mean" in Galton's sense.

Related Calculations

• Multiple Regression — Multiple predictors
• Pearson Correlation — Relationship strength
• R-Squared Calculator — Model fit
• Residual Analysis — Assumption checking
• Prediction Interval Calculator — Uncertainty in predictions

Summary

Simple linear regression fits the line Y = a + bX to model how X predicts Y. The slope (b) represents the change in Y per unit change in X; the intercept (a) is the predicted Y when X = 0. R² indicates the proportion of variance explained. Test the slope for significance and check assumptions using residual plots. Report the equation, R², and significance test. Remember that regression shows association, not causation, and avoid extrapolating beyond your data range.