Multiple linear regression

Multiple regression models the mean response as a linear combination of several predictors, each coefficient describing a change with the others held fixed.

y_i = \beta_0 + \beta_1 x_{i1} + \dots + \beta_k x_{ik} + \varepsilon_i,\qquad \hat{\boldsymbol\beta} = \arg\min \sum e_i^2

Model and least-squares estimates (computed by QR decomposition)

$\beta_j$: change in mean y per unit of $x_j$ , holding the other predictors constant
$k$: number of predictors; p = k + 1 coefficients with the intercept
$\varepsilon_i$: error term, mean 0, variance $\sigma^2$

F = \frac{\text{ESS}/(p-1)}{\hat\sigma^2},\qquad df = (p-1,\ n-p)

Overall F test of H0: all slopes = 0

$\text{ESS}$: explained sum of squares $\sum(\hat y_i-\ybar)^2$
$\hat\sigma^2$: RSS/(n − p), the residual mean square

F = \frac{(\text{RSS}_r - \text{RSS}_f)/(df_r - df_f)}{\text{RSS}_f/df_f}

Nested-model (partial) F test

$\text{RSS}_r,\ \text{RSS}_f$: residual sums of squares of the reduced and full models (same rows)
$df_r,\ df_f$: their residual degrees of freedom

When to use

Predicting y from several predictors, or estimating the association of one predictor with y while adjusting for others.
Categorical predictors enter as dummy (0/1) columns, one per level except a reference level; each dummy's coefficient is the difference in mean y between that level and the reference, other predictors fixed.

Assumptions

LINE conditions on the errors (see residual diagnostics), more observations than coefficients, and no predictor an exact linear combination of others (the engine refuses such "aliased" columns and names them).

How to interpret

"Holding the other predictors constant" is part of every coefficient's meaning; adding or removing predictors can change a coefficient a lot.
The overall F test asks whether the predictors together explain more than chance; the nested F test asks whether a set of extra terms improves the model (for one extra term, F = t²).
Use adjusted R² (not R²) to compare models with different numbers of predictors — see R².

Common mistakes

Interpreting coefficients causally, or "holding constant" combinations that never occur in the data.
Including a dummy for every level plus an intercept (perfect collinearity).
Judging importance by coefficient size when predictors have different units.
Ignoring multicollinearity.

Example

Eight (synthetic) flats: price (thousand SAR) on size (m²) and age (years).

Fit: $\widehat{\text{price}} = 102.7 + 1.917\,\text{size} - 2.642\,\text{age}$ ; $\hat\sigma = 3.88$ , $R^2 = 0.997$ , adjusted $R^2 = 0.996$ .
For flats of the same age, each extra m² goes with 1.92 thousand SAR higher price on average; each extra year of age, with 2.64 lower.
Nested F for adding age to a size-only model: RSS falls from 345.5 to 75.2, $F = 270.4/(75.2/5) = 17.98$ on (1, 5) df, p = 0.0082 — equal to the age t-test ( $t = -4.24$ , $t^2 = 17.98$ ).