Leverage, outliers and Cook's distance

Leverage measures how unusual a point's predictor values are, and Cook's distance measures how much the fitted model changes when the point is removed.

h_i = \mathbf{x}_i^\top (X^\top X)^{-1}\mathbf{x}_i,\qquad \text{simple regression: } h_i = \frac{1}{n} + \frac{(x_i-\xbar)^2}{S_{xx}}

Leverage (hat value)

$h_i$: between 1/n and 1; average value p/n
$\mathbf{x}_i$: row i of the model matrix

D_i = \frac{r_i^2\,h_i}{p\,(1-h_i)}

Cook's distance

$r_i$: internally studentised residual
$p$: number of coefficients including the intercept

When to use

Whenever a few points might be driving a regression: unusual x values, large residuals, or results that seem fragile.

How to interpret

Outlier: large residual (|studentised residual| > about 3).
High leverage: $h_i$ well above the average $p/n$ (rules of thumb: > 2p/n or 3p/n). High leverage is potential, not proof, of influence.
Influential: removing the point changes the coefficients noticeably; Cook's $D_i$ combines residual size and leverage. Values near or above 1 (or clearly larger than the rest) deserve investigation.

Common mistakes

Deleting influential points without a substantive reason (data error, different population). Report results with and without them instead.
Looking only at residuals: a high-leverage point pulls the line towards itself and can have a small residual.
Flagging many points by a fixed cut-off in large data sets without looking at them.

Example

x = 1, 2, 3, 4, 5, 12 and y = 3, 5, 4, 7, 8, 6. The point (12, 6) has leverage $h = 0.892$ (average p/n = 0.33), studentised residual −3.46 and Cook's D = 13.3. With it the slope is 0.21; without it (first five points) the slope is 1.2. One point changes the conclusion.