R², adjusted R² and residual standard error

R² is the share of the response's variation explained by the model, adjusted R² penalises extra predictors, and the residual standard error is the typical size of a residual.

R^2 = 1 - \frac{\text{RSS}}{\text{TSS}},\qquad \text{TSS} = \sum (y_i - \ybar)^2

Coefficient of determination (model with an intercept)

$\text{RSS}$: residual sum of squares $\sum e_i^2$
$\text{TSS}$: total sum of squares about the mean

R^2_{adj} = 1 - (1-R^2)\,\frac{n-1}{n-p}

Adjusted R² (with an intercept)

$n$: number of observations
$p$: number of coefficients including the intercept

\hat\sigma = \sqrt{\frac{\text{RSS}}{n-p}}

Residual standard error

$\hat\sigma$: estimated SD of the errors, in the units of y

When to use

Summarising how well a linear model fits, and (adjusted R², $\hat\sigma$ ) comparing models fitted to the same response.

How to interpret

$R^2 = 0.9$ : the model accounts for 90% of the variation of y around its mean. In simple regression $R^2 = r^2$ .
$\hat\sigma$ says how far observations typically fall from the fitted values, in y's units — often more useful for prediction than $R^2$ .
Adjusted R² only rises when a new predictor improves the fit by more than chance would; it can fall, and can be negative.
Without an intercept, software (and the engine) uses the uncentred $\text{TSS} = \sum y_i^2$ , so that $R^2$ is not comparable with the usual one.

Common mistakes

Treating a high $R^2$ as proof that the model is correct: a curved pattern can have high $R^2$ and clearly patterned residuals.
Treating a low $R^2$ as worthless: a predictor can matter even when y is noisy.
Comparing $R^2$ between models with different responses (e.g. y and log y) or different data.
Adding predictors to raise $R^2$ — it never decreases, even for noise.

Example

For y = 3, 5, 4, 7, 8, 9 on x = 1…6: TSS = 28, RSS = 2.8.

$R^2 = 1 - 2.8/28 = 0.90$ ; $R^2_{adj} = 1 - 0.1 \times 5/4 = 0.875$ .
$\hat\sigma = \sqrt{2.8/4} = 0.837$ sales units.