Statistics Glossary

The probability of rejecting the null hypothesis when it is actually true (Type I error rate). Commonly set at 0.05, meaning a 5% chance of a false positive.

The hypothesis that contradicts the null hypothesis. It represents what the researcher is trying to prove—typically that there is an effect or difference.

Analysis of Variance. A statistical method for comparing means across three or more groups by analyzing the variance within and between groups.

Analysis of Covariance. Combines ANOVA and regression to compare group means while controlling for a continuous covariate.

Correlation of a variable with its own past values. Common in time series data. Violates the independence assumption in standard regression.

A measure for model comparison balancing fit and complexity. Lower AIC indicates a better model. Penalizes additional parameters less than BIC.

The difference in event rates between control and treatment groups. More clinically meaningful than relative measures for assessing treatment impact.

The area under the ROC curve. Ranges from 0.5 (no discrimination) to 1.0 (perfect). Measures overall classification performance.

The proportion of people who develop illness during an outbreak period. Cumulative incidence in epidemic contexts.

The difference in risk between exposed and unexposed groups. The amount of risk attributable to the exposure.

Loss of participants during a study. Differential attrition (different rates by group) can bias results.

Autoregressive Integrated Moving Average. A flexible time series model combining autoregression, differencing, and moving average components.

A time series model where the current value depends linearly on past values. AR(p) uses p lagged values.

The correlation between a time series and its lagged values. Used to identify MA order and check for residual autocorrelation.

A test for unit roots in time series. Null hypothesis: unit root present (non-stationary). Rejection indicates stationarity.

R² adjusted for the number of predictors. Penalizes adding variables that don't improve fit enough. Can decrease when adding weak predictors.

Model probabilities derived from AIC differences. Weight for model i: exp(-ΔAIC/2) / sum over all models.

A goodness-of-fit test emphasizing distribution tails. More sensitive to departures from normality than Kolmogorov-Smirnov.

An approach to statistics that incorporates prior knowledge or beliefs into the analysis, updating these beliefs as new data becomes available using Bayes' theorem.

The probability of failing to reject the null hypothesis when it is actually false (Type II error rate). Statistical power equals 1 - β.

Systematic error that causes estimates to deviate from the true value in a consistent direction. Can arise from sampling methods, measurement, or analysis.

A discrete probability distribution describing the number of successes in a fixed number of independent trials, each with the same probability of success.

A graphical method for comparing two measurement methods. Plots the difference against the mean to assess agreement and identify systematic bias.

Concealing group assignment from participants (single-blind), researchers (double-blind), or both to prevent bias in treatment delivery and outcome assessment.

A formula for updating probabilities based on new evidence. Relates conditional probabilities: P(A|B) = P(B|A) × P(A) / P(B).

A method for controlling family-wise error rate by dividing α by the number of tests. Conservative but simple: α_adjusted = α / n.

A model comparison criterion similar to AIC but with stronger penalty for complexity. Tends to select simpler models than AIC.

A resampling method that estimates sampling distributions by repeatedly sampling with replacement from the original data. Useful for confidence intervals.

A graphical display of the five-number summary. The box spans Q1 to Q3, with a line at the median. Whiskers extend to show data range or outliers.

A continuous distribution bounded between 0 and 1. Useful for modeling proportions and probabilities. Very flexible shape.

A family of power transformations that includes log, square root, and reciprocal. The optimal power parameter is estimated from data.

The ratio of marginal likelihoods for two models. Used for Bayesian model comparison and hypothesis testing.

Model building starting with all predictors, sequentially removing the least significant until all remaining are significant.

Evaluates all possible predictor combinations to find optimal models. Computationally intensive but thorough.

Controls FDR by comparing ordered p-values to linear threshold. Rejects all p-values up to the largest that meets criterion.

Tests equality of variances across groups. Sensitive to non-normality; prefer Levene's test for non-normal data.

A Levene's test variant using median instead of mean. More robust to non-normality.

Mean squared difference between predicted probabilities and binary outcomes. Lower is better; 0 is perfect.

The optimal predictor of random effects given the data. Combines fixed effects with shrunken group-specific estimates.

The balance between underfitting (high bias) and overfitting (high variance). Optimal models minimize total error.

Bootstrap Aggregating. Trains models on bootstrap samples and averages predictions. Reduces variance.

Sequentially training models where each focuses on errors of previous models. Reduces bias.

States that the sampling distribution of the mean approaches a normal distribution as sample size increases, regardless of the population's distribution.

A test statistic used to assess the discrepancy between observed and expected frequencies. Used in goodness-of-fit tests and tests of independence.

A range of values that likely contains the true population parameter. A 95% CI means if we repeated sampling many times, 95% of intervals would contain the true value.

The probability (typically 90%, 95%, or 99%) that a confidence interval contains the true population parameter. Higher confidence requires wider intervals.

A measure of the strength and direction of the linear relationship between two variables. Ranges from -1 (perfect negative) to +1 (perfect positive).

A measure of how two variables change together. Positive covariance indicates they tend to move in the same direction; negative means opposite directions.

A standardized measure of effect size expressing the difference between two means in standard deviation units. Values of 0.2, 0.5, and 0.8 represent small, medium, and large effects.

A measure of internal consistency reliability for scales and questionnaires. Values above 0.7 are generally considered acceptable.

Dividing the population into clusters, randomly selecting some clusters, then sampling all or some members within chosen clusters. Cost-effective for geographically dispersed populations.

Selecting participants based on availability and ease of access rather than random selection. Prone to bias but practical for exploratory research.

An observational study following a group of people over time to see who develops an outcome. Can be prospective (forward in time) or retrospective.

An observational study comparing people with an outcome (cases) to those without (controls), looking back at exposures. Efficient for rare diseases.

A study measuring exposure and outcome at the same point in time. Provides a snapshot but cannot establish temporal sequence or causation.

When a third variable is associated with both the exposure and outcome, distorting the apparent relationship between them. A major threat to causal inference.

The probability of an event occurring given that another event has already occurred. Written as P(A|B), read as "probability of A given B."

A function giving the probability that a random variable is less than or equal to a value. F(x) = P(X ≤ x). Ranges from 0 to 1.

The threshold value that defines the rejection region in hypothesis testing. If the test statistic exceeds this value, reject the null hypothesis.

A technique for assessing model performance by training on subsets of data and testing on held-out portions. K-fold CV divides data into k parts.

The Bayesian analog to confidence intervals. A range that contains the parameter with a specified probability (e.g., 95% credible interval).

The degree to which a test measures the theoretical construct it is intended to measure. Established through convergent and discriminant validity.

A measure describing the center of a distribution. The three main measures are mean, median, and mode, each appropriate for different situations.

Whether a result is large enough to be meaningful in practice. A statistically significant result may not be clinically significant, and vice versa.

An estimator that converges to the true parameter value as sample size increases. Consistency is a desirable large-sample property.

Data that can be divided into distinct groups or categories. Includes nominal (unordered) and ordinal (ordered) types.

Numeric data that can take any value within a range, including decimals. Examples include height, weight, and temperature.

Methods for grouping similar observations into clusters. Types include hierarchical, k-means, and density-based clustering.

A survival analysis method modeling the effect of covariates on hazard rates. Does not require specifying the baseline hazard.

A table showing predicted vs. actual classifications. Contains true positives, false positives, true negatives, and false negatives.

A table displaying frequency distribution of two or more categorical variables. Basis for chi-square tests and measures of association.

A measure of association for nominal variables based on chi-square. Ranges from 0 (no association) to 1 (perfect association).

The proportion of people with a disease who die from it. Indicates disease severity. CFR = Deaths / Cases.

A set of events where at least one must occur. Together they cover all possible outcomes. Probabilities sum to 1.

All outcomes not in an event. The complement of A (written A' or Aᶜ) has probability P(A') = 1 - P(A).

When a measure cannot detect values above a certain point because the upper limit is reached. Leads to clustering at maximum.

Standard deviation expressed as a percentage of the mean. Allows comparison of variability across different scales.

A variable that is controlled for in analysis. In ANCOVA, a continuous variable that may affect the dependent variable.

A measure of influence combining leverage and residual size. Identifies observations that substantially change regression coefficients when removed.

The natural link function for each exponential family distribution. Logit for binomial, log for Poisson, identity for normal.

A plot of autocorrelation or partial autocorrelation against lag. Used to identify appropriate ARIMA orders.

When non-stationary time series share a common stochastic trend, their linear combination is stationary. Basis for error correction models.

A prior distribution that, when combined with a specific likelihood, yields a posterior in the same family. Simplifies Bayesian computation.

Methods to assess whether MCMC chains have converged to the target distribution. Examples: R-hat, trace plots, effective sample size.

Standard errors accounting for correlation within clusters (schools, hospitals, etc.). Observations within clusters are not independent.

How well predicted probabilities match observed frequencies. A well-calibrated model predicting 30% risk should see ~30% events.

The probability that a randomly selected case is ranked higher than a randomly selected non-case. Equals AUC for binary outcomes.

Extension of C-statistic to survival data. Probability that predictions and outcomes are concordant for random pairs.

When one class is much more frequent than others. Causes models to favor the majority class.

Accuracy, precision, recall, F1, specificity derived from the confusion matrix. Each emphasizes different aspects of performance.

The number of independent values that can vary in a calculation. Affects the shape of distributions like t and chi-square.

The spread or variability in a set of data. Common measures include range, variance, standard deviation, and interquartile range.

The number of independent values that can vary in a calculation. Affects the shape of t, chi-square, and F distributions.

Numeric data that can only take specific values, usually whole numbers. Examples include counts (number of children, defects).

A method for classifying observations into predefined groups based on predictor variables. Also identifies variables that best separate groups.

A method comparing changes over time between treatment and control groups. Assumes parallel trends in absence of treatment.

The ratio of variance under complex sampling to variance under simple random sampling. Accounts for clustering and stratification effects.

A smoothed version of a histogram showing the probability density function. Uses kernel density estimation.

When the magnitude of an effect changes with the amount of exposure. Provides evidence supporting causation.

The outcome variable that is measured or predicted. Also called response variable or Y variable in regression.

The number of independent values that can vary in a calculation. Affects the shape of t, chi-square, and F distributions.

Measures how much each regression coefficient changes when an observation is deleted. Scaled by standard error for comparability.

Measures how much the fitted value changes when an observation is deleted. Combines leverage and residual information.

Tests for first-order autocorrelation in residuals. Values near 2 indicate no autocorrelation; near 0 or 4 indicate positive or negative autocorrelation.

A measure of model fit in GLMs. Twice the difference between the log-likelihood of the saturated model and the fitted model.

The signed square root of each observation's contribution to deviance. Preferred residuals for GLMs; should be approximately normal.

A scale parameter in GLMs relating variance to mean. Fixed at 1 for Poisson and binomial; estimated for normal and gamma.

Subtracting consecutive observations to remove trends and achieve stationarity. First difference: Yₜ - Yₜ₋₁.

A Bayesian model comparison measure. Like AIC but uses posterior mean deviance and effective number of parameters.

A multiple comparison procedure for comparing several treatments to a single control. More powerful than Bonferroni for this purpose.

A model's ability to distinguish between outcomes. Measured by AUC/c-statistic. Good discrimination ≠ good calibration.

A model that recursively splits data based on feature values. Interpretable but prone to overfitting.

When information from outside the training set improperly influences model building. Leads to overoptimistic performance.

A standardized measure of the magnitude of an effect or relationship, independent of sample size. Examples include Cohen's d, r, and η².

A study where researchers actively manipulate one or more variables to observe effects. Stronger for establishing causation than observational studies.

The long-run average value of a random variable over many repetitions. The probability-weighted average of all possible values.

A continuous distribution modeling time between events in a Poisson process. Memoryless property: future waiting time independent of time already waited.

An effect size measure for ANOVA representing the proportion of total variance explained by the factor. Tends to overestimate population effect.

The degree to which study results can be generalized to other populations, settings, or times. Threatened by non-representative samples.

A rule or formula for calculating an estimate of a population parameter from sample data. Properties include bias, consistency, and efficiency.

In PCA and factor analysis, a measure of variance explained by each component or factor. Larger eigenvalues indicate more important components.

The count expected in each cell of a contingency table if variables are independent. Compared to observed frequencies in chi-square tests.

Incorrectly inferring individual-level relationships from group-level (aggregate) data. A form of erroneous reasoning.

A class of probability distributions including normal, binomial, Poisson, gamma, and exponential. Foundation for GLMs.

A forecasting method giving exponentially decreasing weights to older observations. Includes simple, Holt, and Holt-Winters variants.

In MCMC, the equivalent number of independent samples given autocorrelation in the chain. ESS < actual samples due to dependence.

An approach estimating prior parameters from data. In mixed models, used to estimate random effects (BLUPs).

A regularization method combining L1 (lasso) and L2 (ridge) penalties. Balances variable selection and coefficient shrinkage.

Combining multiple models to improve predictions. Includes bagging, boosting, and stacking.

A measure of impurity or uncertainty in classification. Maximum when classes are equally distributed.

A probability distribution used in ANOVA and regression analysis. It's the ratio of two chi-square distributions divided by their degrees of freedom.

The test statistic in ANOVA, calculated as the ratio of between-group variance to within-group variance. Larger values suggest group means differ.

A nonparametric alternative to repeated-measures ANOVA. Compares three or more related groups using ranks rather than raw values.

The expected proportion of false positives among all significant results. Less conservative than family-wise error control. Benjamini-Hochberg procedure controls FDR.

A set of five values summarizing a distribution: minimum, first quartile (Q1), median (Q2), third quartile (Q3), and maximum. Basis for box plots.

The number of times a value or category occurs in a dataset. Relative frequency expresses this as a proportion or percentage of total observations.

A meta-analysis model assuming all studies estimate the same true effect. Appropriate when studies are functionally identical.

A method for identifying latent variables (factors) that explain correlations among observed variables. Common in psychology and social sciences.

The harmonic mean of precision and recall. Balances both metrics in a single score. F1 = 2 × (precision × recall) / (precision + recall).

An experimental design studying all combinations of two or more factors. Allows testing main effects and interactions efficiently.

An exact test for association in 2×2 tables. Preferred over chi-square when expected frequencies are small (< 5).

A graphical display of effect estimates from multiple studies in a meta-analysis. Shows individual and pooled estimates with confidence intervals.

A scatter plot of effect size vs. study precision in meta-analysis. Asymmetry suggests publication bias.

When a measure cannot detect values below a certain point because the lower limit is reached. Leads to clustering at minimum.

Model building starting with no predictors, sequentially adding the most significant variable until no improvement.

The probability of making at least one Type I error among all tests. Controlled by Bonferroni, Holm, and similar methods.

In mixed models, effects that are constant across all groups or levels. The population-average effect of a predictor.

Choosing which predictors to include in a model. Methods include filter, wrapper, and embedded approaches.

How well a statistical model fits the observed data. Assessed through measures like R², chi-square tests, or visual inspection of residuals.

A continuous distribution for positive values, often used for waiting times and reliability. Characterized by shape and rate parameters.

An effect size using only the control group standard deviation. Appropriate when treatment affects variability or groups have unequal variances.

The nth root of the product of n values. Appropriate for ratios, growth rates, and log-normally distributed data.

A flexible extension of linear regression allowing non-normal response distributions and nonlinear relationships via a link function.

Tests whether one time series helps predict another. X "Granger causes" Y if past X values improve Y predictions beyond past Y alone.

An MCMC algorithm that samples each parameter conditionally on current values of others. Efficient when conditional distributions are known.

A post-hoc test for unequal variances and sample sizes. Does not assume homogeneity of variance.

A method for analyzing correlated data focusing on population-average effects. Robust to correlation structure misspecification.

A model's ability to perform well on new, unseen data. The ultimate goal of predictive modeling.

Exhaustively searching a specified parameter grid to find optimal hyperparameters. Can be computationally expensive.

A boosting method fitting new models to the gradient of the loss function. XGBoost and LightGBM are popular implementations.

A measure of misclassification probability used in decision trees. Ranges from 0 (pure) to 0.5 (binary, equally split).

A formal procedure for deciding whether sample data provides sufficient evidence to reject a null hypothesis in favor of an alternative hypothesis.

When the variance of residuals is not constant across levels of the predictor. Violates a key regression assumption and affects inference.

When the variance of residuals is constant across all levels of the predictor. A key assumption of ordinary least squares regression.

A graphical display of the distribution of continuous data using bars. Bar heights represent frequency or proportion of observations in each bin.

Variation in study results beyond what would be expected by chance. In meta-analysis, indicates that studies may not all estimate the same effect.

The instantaneous rate of event occurrence at time t, given survival to that time. Central concept in survival analysis.

The ratio of hazard rates between two groups. HR > 1 means higher risk in the first group; HR < 1 means lower risk.

A bias-corrected version of Cohen's d for small samples. Multiplies d by a correction factor based on sample size.

When participants modify behavior because they know they're being observed. A threat to external validity.

The reciprocal of the arithmetic mean of reciprocals. Appropriate for rates and ratios.

The matrix H that maps observed Y values to fitted values: Ŷ = HY. Diagonal elements (hᵢᵢ) are leverages.

A two-part model: first models zero vs. non-zero, then models positive counts. All zeros come from one process unlike zero-inflated models.

The narrowest credible interval containing the specified probability. Every point inside has higher posterior density than outside.

The simplest validation: split data into training and test sets. Less reliable than cross-validation but computationally faster.

A loss function that is quadratic for small residuals and linear for large ones. Less sensitive to outliers than squared error.

A step-down procedure for multiple testing. More powerful than Bonferroni while still controlling FWER.

A step-up multiple testing procedure. More powerful than Holm when tests are independent but requires independence assumption.

Tests logistic regression calibration by comparing observed and expected events across risk groups.

A parameter set before training (e.g., regularization strength, tree depth). Tuned via cross-validation or grid search.

The range between the 25th and 75th percentiles, representing the middle 50% of data. A robust measure of spread less affected by outliers.

A measure of reliability and agreement for measurements made by multiple raters or on multiple occasions. Used to assess inter-rater reliability.

Two events are independent if the occurrence of one does not affect the probability of the other. P(A|B) = P(A) when A and B are independent.

The predicted value of the dependent variable when all independent variables equal zero. The point where the regression line crosses the y-axis.

The degree to which a study establishes a causal relationship between treatment and outcome. Threatened by confounding, bias, and poor design.

Numeric data with meaningful intervals between values but no true zero point (e.g., temperature in Celsius, IQ scores).

The rate of new cases of a condition occurring over time. Measures risk of developing the condition.

Replacing missing values with estimated values. Methods range from simple (mean imputation) to sophisticated (multiple imputation).

A variable that affects treatment but only influences outcome through treatment. Used to estimate causal effects when confounding exists.

Analyzing participants in their assigned groups regardless of actual treatment received. Preserves randomization and reflects real-world effectiveness.

When the effect of one factor depends on the level of another factor. Tested in factorial ANOVA and regression with interaction terms.

A range of values that likely contains the parameter. Confidence intervals are the most common form.

A variable manipulated or used to predict the outcome. Also called predictor, explanatory variable, or X variable.

A prior incorporating substantial prior knowledge about parameters. Influences posterior more than non-informative priors.

A measure balancing model fit and complexity for model comparison. Lower values indicate better models. Examples: AIC, BIC, DIC.

The reduction in entropy (or Gini impurity) achieved by splitting on a feature. Used to build decision trees.

The probability of two or more events occurring together. For independent events, it equals the product of individual probabilities.

A non-informative prior derived from the Fisher information. Invariant under reparameterization.

A resampling method that systematically leaves out one observation at a time. Used for bias estimation and variance estimation.

Tests normality using skewness and kurtosis. Under normality, both should be near expected values.

A measure of the "tailedness" of a distribution. High kurtosis indicates heavy tails and outliers; low kurtosis indicates light tails.

A nonparametric alternative to one-way ANOVA. Compares three or more independent groups when the normality assumption is violated.

A nonparametric method for estimating survival probabilities over time, accounting for censored observations. Produces the familiar survival curve.

A nonparametric measure of correlation based on concordant and discordant pairs. More robust than Spearman for small samples with many ties.

Kwiatkowski-Phillips-Schmidt-Shin test. Tests stationarity with opposite null to ADF: null is stationarity, alternative is unit root.

Divides data into k subsets, trains on k-1 folds, tests on remaining fold, rotates. Common choice: k=5 or k=10.

Tests whether data follows a specified distribution by comparing empirical and theoretical CDFs. Can compare two samples.

The probability of observing the data given specific parameter values. Central to maximum likelihood estimation and Bayesian inference.

A method for modeling the relationship between a dependent variable and one or more independent variables using a straight line.

A test of homogeneity of variances across groups. Less sensitive to departures from normality than Bartlett's test.

A study collecting data from the same subjects repeatedly over time. Can detect changes and temporal relationships but is time-consuming and expensive.

As sample size increases, the sample mean converges to the population mean. Foundation for why larger samples give more reliable estimates.

A regression method for binary outcomes that models the log-odds of the outcome as a linear function of predictors. Produces odds ratios.

A method for fitting models by minimizing the sum of squared residuals. The basis for ordinary least squares (OLS) regression.

Excluding any case with missing data on any variable. Simple but can greatly reduce sample size and introduce bias if data is not MCAR.

A hypothesis test comparing survival distributions between two or more groups. The most common test for survival curve differences.

When the event occurred before the observation period began, so the exact time is unknown. Less common than right censoring.

A distribution where the logarithm of the variable is normally distributed. Common for inherently positive quantities like income or concentrations.

A regularized regression with penalty on absolute coefficients (L1). Can shrink some coefficients exactly to zero, performing variable selection.

Applying logarithm to reduce right skewness and stabilize variance. Common for ratios, concentrations, and multiplicative processes.

A variable that is not directly observed but inferred from other measured variables. Examples include intelligence, attitudes.

A measure of how far an observation's predictor values are from the mean. High leverage points can strongly influence the regression line.

In GLMs, the function connecting the linear predictor to the mean of the response. Examples: logit for binary, log for counts, identity for normal.

A past value of a time series. Lag 1 is the previous observation, lag 2 is two periods back, etc.

Tests whether autocorrelations of residuals are collectively zero. Used to check if time series model adequately captures patterns.

Cross-validation leaving out one observation at a time. PSIS-LOO approximates this efficiently for Bayesian models.

A Kolmogorov-Smirnov test modified for testing normality when parameters are estimated from data.

The logarithm of the likelihood function. Summing log-likelihoods is numerically more stable than multiplying likelihoods.

Compares nested models using -2 times the difference in log-likelihoods. Follows chi-square distribution with df = difference in parameters.

The range of values above and below a sample statistic within a confidence interval. Decreases with larger samples.

The arithmetic average of a set of values. Calculated by summing all values and dividing by the count. Sensitive to outliers.

The middle value when data is ordered from smallest to largest. More robust to outliers than the mean.

The most frequently occurring value in a dataset. A distribution can have no mode, one mode (unimodal), or multiple modes.

A nonparametric test comparing two independent groups. Does not assume normality; tests whether one group tends to have larger values than the other.

A test for paired nominal data, specifically 2×2 tables with matched pairs. Assesses whether the row and column marginal frequencies are equal.

The probability of a single event regardless of other variables. Obtained by summing or integrating joint probabilities over other variables.

The problem of increased false positive rate when performing many statistical tests. If you test 20 hypotheses at α=0.05, expect 1 false positive by chance.

When predictor variables in a regression are highly correlated with each other. Inflates standard errors and makes coefficients unstable.

Linear regression with two or more predictor variables. Allows examining the effect of each predictor while controlling for others.

A method for estimating parameters by finding values that maximize the probability of observing the data. Produces asymptotically efficient estimators.

A statistical method for combining results from multiple studies to obtain a more precise estimate of an effect. Increases statistical power.

A calculation that smooths time series data by averaging consecutive observations. Reduces noise and highlights trends.

Multivariate Analysis of Variance. Extends ANOVA to multiple dependent variables simultaneously, testing whether groups differ on a combination of outcomes.

Missing data mechanism where missingness is unrelated to any variables. The strongest and often unrealistic assumption.

Missing data mechanism where missingness depends on observed variables but not the missing values themselves. Allows valid analysis with proper methods.

Missing data mechanism where missingness depends on the unobserved values. Most problematic; requires specialized methods or sensitivity analysis.

Creating several plausible imputed datasets, analyzing each, and combining results. Properly accounts for uncertainty due to missing data.

Sum of squares divided by degrees of freedom. The F-statistic in ANOVA is the ratio of mean squares: F = MSB / MSW.

The overall effect of one factor averaged across levels of other factors. Distinct from interaction effects in factorial designs.

The number of deaths in a population over a time period divided by population size. Can be crude or specific (age, cause).

Events that cannot occur simultaneously. If A occurs, B cannot occur. P(A and B) = 0 for mutually exclusive events.

Inaccuracy in measuring variables. Can be random (reduces precision) or systematic (introduces bias).

A variable that affects the strength or direction of the relationship between two other variables. Creates interaction effects.

A variable that explains how or why an effect occurs. The independent variable affects the mediator, which then affects the outcome.

A time series model where the current value depends on past error terms. MA(q) uses q lagged errors.

A class of algorithms for sampling from probability distributions. Used in Bayesian inference when posteriors lack closed forms.

A general MCMC algorithm using a proposal distribution and acceptance probability. Foundation for many Bayesian sampling methods.

The probability of data averaged over all parameter values weighted by the prior. Also called evidence or integrated likelihood.

A model selection criterion balancing fit and complexity. Good models have Cp near p (number of parameters).

Combining predictions or estimates from multiple models weighted by their probability or performance. Accounts for model uncertainty.

A class of robust estimators that minimize a function of the residuals. Includes maximum likelihood and robust regression.

A pseudo R-squared comparing model log-likelihood to null model. Values 0.2-0.4 indicate excellent fit.

A model with both fixed effects (population parameters) and random effects (group-specific deviations). Also called multilevel or hierarchical model.

A mixed model for hierarchically structured data (students within schools, patients within hospitals). Accounts for clustering.

A symmetric, bell-shaped probability distribution characterized by its mean and standard deviation. Many statistical methods assume normality.

The default assumption that there is no effect or no difference. Statistical tests assess evidence against this hypothesis.

A discrete probability distribution modeling the number of failures before a specified number of successes. Useful for overdispersed count data.

Methods that do not assume a specific probability distribution for the data. Based on ranks or resampling. More robust but often less powerful.

The average number of patients who need to receive a treatment for one additional patient to benefit. NNT = 1 / absolute risk reduction.

Categorical data with no inherent order. Categories are names or labels only (e.g., blood type, gender, country).

The probability that a negative test result is truly negative. Depends on prevalence of the condition.

Bias occurring when non-respondents differ systematically from respondents. Threatens external validity of survey results.

Scaling data to a specific range (often 0-1). Different from standardization; preserves the shape of the distribution.

A GLM for overdispersed count data where variance exceeds the mean. Adds a dispersion parameter to the Poisson model.

A prior expressing minimal prior knowledge, letting data dominate inference. Examples: flat prior, Jeffreys prior.

A pseudo R-squared rescaled to range from 0 to 1. Adjusts Cox-Snell R² to achieve maximum of 1.

The ratio of odds of an event occurring in one group versus another. Common in case-control studies and logistic regression.

A data point that differs significantly from other observations. May indicate measurement error, data entry error, or genuine extreme values.

A study where researchers observe subjects without manipulating variables. Cannot prove causation but useful when experiments are impractical or unethical.

A hypothesis test where the alternative hypothesis specifies a direction (greater than or less than). More powerful for detecting effects in one direction.

When a model fits the training data too closely, capturing noise rather than signal. Leads to poor performance on new data.

A less biased effect size for ANOVA than eta-squared. Provides a better estimate of population effect size, especially for small samples.

Categorical data with a meaningful order but unequal intervals between categories (e.g., education level, pain severity rating).

When observed variance exceeds the variance expected under the assumed model. Common in count data; addressed by quasi-likelihood or negative binomial.

A predictor with coefficient fixed at 1, used in Poisson regression to model rates by including log(exposure) as offset.

The probability of obtaining results at least as extreme as observed, assuming the null hypothesis is true. Smaller values provide stronger evidence against H₀.

A value below which a given percentage of observations fall. The 90th percentile means 90% of values are below it.

A discrete probability distribution expressing the probability of events occurring in a fixed interval, given a constant mean rate.

The complete set of items or individuals of interest in a statistical study. Usually too large to measure entirely, so we use samples.

The probability of correctly rejecting a false null hypothesis (1 - β). Higher power means better ability to detect true effects.

A statistical test comparing two related means, typically before-after measurements on the same subjects. Tests whether the mean difference differs from zero.

A correlation coefficient between a continuous variable and a binary variable. Mathematically equivalent to the Pearson correlation in this case.

An inactive treatment given to control groups to account for the psychological effects of receiving treatment. Enables blinding in clinical trials.

For continuous random variables, a function describing relative likelihood of different values. Area under the curve between two points gives probability.

For discrete random variables, a function giving the probability of each possible value. Probabilities must sum to 1.

Follow-up tests conducted after ANOVA finds a significant difference, to determine which specific groups differ. Examples: Tukey HSD, Bonferroni, Scheffé.

In Bayesian statistics, the probability distribution representing beliefs about a parameter before observing data. Combined with likelihood to form the posterior.

In Bayesian statistics, the updated probability distribution for a parameter after observing data. Combines prior beliefs with evidence from data.

A resampling method that tests hypotheses by randomly rearranging group labels. Makes minimal distributional assumptions.

The tendency for studies with significant or positive results to be more likely published. Can bias meta-analyses and systematic reviews.

A numerical characteristic of a population, such as the population mean (μ) or standard deviation (σ). Usually unknown and estimated from samples.

The proportion of a population with a condition at a specific time. A snapshot measure of disease burden.

The probability that a positive test result is truly positive. Depends on prevalence of the condition.

A dimensionality reduction technique that transforms correlated variables into uncorrelated principal components capturing maximum variance.

The proportion of positive predictions that are correct. Also called positive predictive value. Precision = TP / (TP + FP).

The probability of receiving treatment given observed covariates. Used in observational studies to reduce confounding through matching or weighting.

Analyzing only participants who completed treatment as assigned. Shows efficacy under ideal conditions but can introduce bias.

Regression including polynomial terms (x², x³, etc.) to model curvilinear relationships while remaining linear in the coefficients.

A measure of association for 2×2 tables. Equivalent to Pearson correlation for two binary variables. Ranges from -1 to 1.

The correlation between two variables after controlling for the effects of one or more other variables.

Improvement in outcomes due to belief in treatment rather than the treatment itself. Controlled for using placebo groups.

A single value estimate of a population parameter. Examples include sample mean, proportion, and correlation coefficient.

A GLM for count data assuming the response follows a Poisson distribution. Uses log link; coefficients represent log rate ratios.

The difference between observed and fitted values divided by the estimated standard deviation. Raw residual standardized by variance.

The correlation between a time series and its lag after removing the effect of intervening lags. Used to identify AR order.

The probability of a hypothesis or parameter value after observing data. Combines prior probability with likelihood.

The distribution of future observations given observed data. Integrates over parameter uncertainty.

The distribution of data implied by the prior before observing data. Used for prior checking and model criticism.

Predicted Residual Error Sum of Squares. Sum of squared leave-one-out prediction errors. Assesses predictive ability.

Measures of fit for logistic regression analogous to R². Not directly interpretable as variance explained. Examples: McFadden, Cox-Snell.

Values that divide ordered data into four equal parts. Q1 (25th percentile), Q2 (median), and Q3 (75th percentile).

A value below which a given proportion of observations fall. The inverse of the CDF. Quartiles (25%, 50%, 75%) are specific quantiles.

Quantile-Quantile plot comparing data quantiles to theoretical distribution quantiles. Points on a straight line indicate good fit.

An estimation approach requiring only mean-variance relationship, not full distribution. Accommodates overdispersion by estimating dispersion.

The minimum FDR at which a test would be called significant. FDR analog of the p-value.

The proportion of variance in the dependent variable explained by the independent variable(s). Ranges from 0 to 1.

A variable whose possible values are outcomes of a random phenomenon. Can be discrete (countable) or continuous (any value in a range).

Statistical methods for estimating relationships between variables. Used to predict values and understand variable associations.

The ratio of probabilities of an event in two groups. A RR of 2 means the event is twice as likely in the exposed group.

A sampling method where every member of the population has an equal probability of being selected. The gold standard for obtaining representative samples.

An experimental study where participants are randomly assigned to treatment or control groups. The gold standard for establishing causal relationships.

The difference between an observed value and the value predicted by a model. Analyzing residuals helps assess model fit and assumptions.

The position of a value when data is ordered from smallest to largest. Used in nonparametric tests like Mann-Whitney and Spearman correlation.

The absolute difference in risk between two groups. Same as absolute risk reduction but can be positive or negative depending on comparison direction.

The consistency of a measurement. A reliable measure produces similar results under consistent conditions. Necessary but not sufficient for validity.

The difference between the maximum and minimum values in a dataset. Simple but sensitive to outliers.

A meta-analysis model assuming true effects vary across studies. Accounts for between-study heterogeneity. Generally more conservative.

Numeric data with meaningful intervals and a true zero point, allowing meaningful ratios (e.g., height, weight, income).

A time series where each value equals the previous value plus a random error. Stock prices often modeled as random walks.

When the event has not occurred by the end of observation. The most common type of censoring in survival analysis.

Receiver Operating Characteristic curve. Plots sensitivity vs. (1-specificity) at various thresholds. Shows tradeoff between true and false positives.

The proportion of actual positives correctly identified. Same as sensitivity. Recall = TP / (TP + FN).

A causal inference method exploiting sharp cutoffs in treatment assignment. Compares outcomes just above and below the threshold.

Systematic error from how questions are asked or answered. Includes social desirability bias, acquiescence bias, and recall bias.

A regularized regression that adds a penalty on squared coefficients (L2). Shrinks coefficients toward zero to reduce overfitting.

Adding a penalty term to prevent overfitting by constraining model complexity. Common in regression and machine learning.

A graph of residuals against predicted values or predictors. Used to check regression assumptions like linearity and homoscedasticity.

The tendency for extreme values to be followed by less extreme values. Can be mistaken for treatment effects.

Methods that perform well even when assumptions are violated or outliers are present. Examples include median and trimmed mean.

A convergence diagnostic comparing within-chain and between-chain variance. Values near 1 suggest convergence; >1.1 indicates problems.

Standard errors that remain valid under heteroscedasticity or other assumption violations. Typically larger than model-based SEs.

Tests randomness by counting runs (sequences of similar values). Too few or too many runs suggests non-randomness.

In mixed models, effects that vary across groups drawn from a population. Assumed normally distributed around zero.

An estimation method for mixed models that gives unbiased variance estimates. Preferred over ML for variance components.

A mixed model where the intercept varies across groups but slopes are fixed. The simplest random effects structure.

A mixed model where both intercept and one or more slopes vary across groups. Allows effect sizes to differ by group.

An ensemble of decision trees trained on bootstrap samples with random feature subsets. Robust and widely applicable.

A subset of the population selected for study. Good sampling methods ensure the sample represents the population.

The number of observations in a sample. Larger samples generally provide more precise estimates and more powerful tests.

The probability distribution of a statistic (like the mean) calculated from all possible samples of a given size.

The threshold (α) below which the p-value leads to rejecting the null hypothesis. Common values are 0.05, 0.01, and 0.10.

A measure of asymmetry in a distribution. Positive skew has a long right tail; negative skew has a long left tail.

A measure of the spread of data around the mean. The square root of variance. In a normal distribution, ~68% of data falls within one SD.

The standard deviation of a sampling distribution. Measures the precision of a sample statistic as an estimate of the population parameter.

A nonparametric measure of rank correlation. Assesses how well the relationship between two variables can be described by a monotonic function.

The simplest nonparametric test for paired data. Counts whether differences go in the positive or negative direction, ignoring magnitudes.

A test of normality that assesses whether a sample comes from a normally distributed population. One of the most powerful normality tests.

Dividing the population into subgroups (strata) and randomly sampling from each stratum. Ensures representation of all subgroups.

Selecting every kth element from a list after a random starting point. Simpler than random sampling but can introduce bias if the list has periodic patterns.

Systematic error arising from how participants are selected or retained in a study. Leads to samples that do not represent the target population.

Bias occurring when some members of the population are more likely to be included in the sample than others. Threatens generalizability of results.

The change in the dependent variable for a one-unit change in the independent variable. Indicates direction and strength of the relationship.

A graph showing the relationship between two continuous variables. Each point represents one observation. Useful for assessing correlation and patterns.

When a result is unlikely to have occurred by chance alone, typically p < 0.05. Does not indicate practical importance or effect size.

A numerical value calculated from sample data, such as the sample mean (x̄) or standard deviation (s). Used to estimate population parameters.

The proportion of actual positives correctly identified by a test. Also called true positive rate or recall.

The proportion of actual negatives correctly identified by a test. Also called true negative rate.

A time series property where statistical properties (mean, variance) remain constant over time. Many methods require stationary data.

Regular, predictable patterns in time series that repeat over fixed periods (daily, weekly, yearly). Distinct from irregular fluctuations.

The list or source from which a sample is drawn. Incomplete frames cause coverage error if some population members are excluded.

The sum of squared deviations from a mean. Partitioned into between-group (SSB) and within-group (SSW) components in ANOVA.

The correlation between Y and the part of X not explained by other variables. Also called part correlation.

The set of all possible outcomes of a random experiment. Denoted by S or Ω. Events are subsets of the sample space.

A transformation that reduces right skewness and stabilizes variance for count data. Milder than log transformation.

Converting values to z-scores by subtracting the mean and dividing by standard deviation. Results in mean=0, SD=1.

The difference between a sample statistic and the true population parameter due to random variation in sampling.

When a trend present in subgroups reverses or disappears when groups are combined. Demonstrates the importance of stratification.

Drawing conclusions about populations based on sample data. Includes estimation and hypothesis testing.

A residual divided by an estimate of its standard deviation that excludes the current observation. Follows t-distribution under normality.

A residual divided by its estimated standard error. Values beyond ±2 or ±3 suggest potential outliers.

A model building strategy that adds or removes predictors based on statistical criteria. Combines forward selection and backward elimination.

A robust variance estimator valid even when model assumptions are violated. Also called Huber-White or heteroscedasticity-consistent estimator.

A conservative post-hoc test allowing any contrast. Controls FWER for all possible comparisons, not just pairwise.

Tests parameters using the score function evaluated at the null hypothesis. Also called Lagrange multiplier test.

In mixed models, random effects are pulled toward zero, especially for small groups. Balances group-specific and population estimates.

Synthetic Minority Over-sampling Technique. Creates synthetic examples of the minority class to address imbalance.

A probability distribution similar to normal but with heavier tails. Used when sample size is small or population variance is unknown.

A hypothesis test comparing means. One-sample compares to a known value; two-sample compares two groups; paired compares before-after.

Rejecting the null hypothesis when it is actually true (false positive). The probability is α, the significance level.

Failing to reject the null hypothesis when it is actually false (false negative). The probability is β, related to statistical power.

A statistical test comparing means of two independent groups. Assumes normally distributed data and tests whether the population means differ.