Regression analysis

Regression analysis estimates how an outcome varies with one or more explanatory variables. It can describe associations, support prediction and—under a credible research design—help estimate causal effects. The calculation alone does not turn an association between price and demand, or any other pair of variables, into causation.

When to use it

Use regression when a decision requires a quantified relationship, a conditional prediction or a test of how an outcome differs as selected inputs change.

It can help investigate questions such as:

• How are loyalty and satisfaction associated with profitability after accounting for relevant differences?
• Does brand perception predict sales beyond price, distribution and seasonality?
• Which dimensions of product quality are associated with customer satisfaction?
• Is employee engagement related to retention, and what alternative explanations must be considered?

Regression is appropriate only when the variables can be measured meaningfully, the sample supports the model and the assumptions fit the intended inference. For causal questions, design matters more than statistical significance.

Origins

Regression emerged from nineteenth-century work in statistics. Francis Galton introduced the idea of “regression toward mediocrity” while studying inherited characteristics; Karl Pearson formalised correlation and regression, while Adrien-Marie Legendre and Carl Friedrich Gauss contributed the least-squares method used in linear models. The technique later became foundational in economics, science, social research and policy analysis.

What it is

A regression expresses a dependent variable as a function of one or more independent variables plus unexplained variation. In a simple linear case, the slope estimates the average change in the outcome associated with a change in the predictor.

Correlation summarises the direction and strength of association between variables symmetrically. Regression assigns different roles to predictors and outcome, estimates conditional relationships and can include several predictors. Neither method, without additional assumptions or design, proves what causes what.

A model can be used for explanation or prediction, but those goals require different validation. A coefficient may be statistically distinguishable from zero yet too small to matter, and a model that fits historical data may predict new cases poorly. In legal, government and business settings, results must therefore be interpreted with effect sizes, uncertainty, model assumptions and the provenance of the data.