AIC & BIC Calculator

Use AIC and BIC when comparing statistical models to find the best balance between fit and complexity. Both penalize models for having more parameters, but BIC penalizes more heavily, favoring simpler models.

What Are AIC and BIC?

Akaike Information Criterion (AIC)

$AIC = 2k - 2\ln(L)$

Where:

• k = number of parameters
• L = maximum likelihood

Lower AIC = better model.

Bayesian Information Criterion (BIC)

$BIC = k\ln(n) - 2\ln(L)$

Where:

• k = number of parameters
• n = sample size
• L = maximum likelihood

Lower BIC = better model.

Why Penalize Complexity?

Without penalties, adding more parameters always improves fit. But:

• Overly complex models overfit
• Extra parameters may capture noise
• Simple models generalize better

AIC and BIC balance fit (likelihood) against complexity (parameters).

AIC vs. BIC

When BIC penalty > AIC penalty:

• n > e² ≈ 7.4 (almost always)
• Difference grows with sample size

Example Comparison

Three regression models on n = 100:

AIC selects: Model 2 (AIC = 250) BIC selects: Model 2 (BIC = 263.0)

Both agree here, but with larger k, BIC would penalize more.

Interpreting Differences

For AIC

For BIC

Similar interpretation, though BIC differences are often larger due to heavier penalty.

Model Selection Rules

Using AIC:

1. Calculate AIC for all candidate models
2. Select model with lowest AIC
3. Report AIC for top candidates
4. Consider models within ΔAIC < 2 as competitive

Using BIC:

1. Calculate BIC for all candidate models
2. Select model with lowest BIC
3. BIC tends to prefer simpler models

Akaike Weights

Convert AIC to model probabilities:

$w_i = \frac{\exp(-\Delta AIC_i / 2)}{\sum_j \exp(-\Delta AIC_j / 2)}$

Where ΔAIC_i = AIC_i - AIC_min.

Interpretation: w_i is the probability that model i is the best model given the data and candidate set.

Corrected AIC (AICc)

For small samples, use the corrected version:

$AIC_c = AIC + \frac{2k(k+1)}{n-k-1}$

Use AICc when: n/k < 40

AICc reduces to AIC as n → ∞.

When to Use Which

Philosophical difference:

• AIC: Best model for prediction (may not be "true")
• BIC: Consistent estimator of "true" model order

Calculating from Software Output

From Residual Sum of Squares (Regression)

$AIC = n\ln(RSS/n) + 2k$ $BIC = n\ln(RSS/n) + k\ln(n)$

From R-squared

$AIC = n\ln(1-R^2) + 2k + C$

Where C is a constant (doesn't matter for comparison).

Nested vs. Non-Nested Models

Nested models: One is a special case of another.

• Can also use likelihood ratio test
• AIC/BIC work fine

Non-nested models: Different structural forms.

• Can't use likelihood ratio test
• AIC/BIC are ideal for this

Applications

Regression

Selecting which predictors to include:

• Compare models with different variables
• Balance fit against number of predictors

Time Series

Choosing ARIMA order:

• Compare (p,d,q) combinations
• Select based on AIC or BIC

Mixed Models

Comparing random effect structures:

• Different covariance structures
• Which random effects to include

Machine Learning

Model complexity tuning:

• Regularization strength
• Number of components (PCA)

Common Mistakes

Comparing Across Different Data

AIC/BIC only compare models fit to the same data. Different sample sizes or variables invalidate comparison.

Ignoring Model Assumptions

Low AIC doesn't validate assumptions. Check residuals regardless of which model wins.

Over-relying on One Criterion

Consider both AIC and BIC, plus domain knowledge. If they disagree substantially, investigate why.

Using for Hypothesis Testing

AIC/BIC are for model selection, not hypothesis testing. They don't give p-values.

AIC/BIC vs. Cross-Validation

For large models or complex data, cross-validation may be more reliable but takes longer.

Stepwise Selection Issues

Using AIC/BIC in stepwise procedures:

• Doesn't explore all possible models
• Can miss optimal solution
• Better: All-subsets regression when feasible

Reporting Guidelines

Standard format: "Model 2 had the lowest AIC (AIC = 250) compared to Model 1 (AIC = 256) and Model 3 (AIC = 251). BIC also selected Model 2 (BIC = 263.0). Model 2 included predictors age, treatment, and baseline score."

Include:

• AIC/BIC for top candidate models
• Which criterion used and why
• Final model specification
• Model diagnostics

Historical Context

Hirotugu Akaike developed AIC in 1973, connecting information theory to statistical model selection. He showed that minimizing AIC approximately minimizes the Kullback-Leibler divergence.

Gideon Schwarz developed BIC in 1978 from a Bayesian perspective. BIC approximates the Bayes factor between models.

Both criteria revolutionized model selection by providing objective, likelihood-based comparison methods.

Related Calculations

• R-Squared Calculator — Model fit
• Multiple Regression — Model building
• Linear Regression — Simple models
• F-Distribution — Nested model tests

Summary

AIC and BIC compare models by balancing fit (likelihood) against complexity (parameters). Lower values indicate better models. AIC uses a fixed penalty (2k); BIC's penalty grows with sample size (k ln n). Use AIC for prediction, BIC when seeking the "true" model. For small samples, use AICc. Compare only models fit to identical data. Report both criteria for major comparisons and consider models within ΔAIC < 2 as competitive.