Inference for regression coefficients

Each coefficient has a standard error, a t-test of β = 0 and a confidence interval on n − p degrees of freedom.

t = \frac{b_j}{\SE(b_j)},\qquad df = n - p

t-test of H0: βⱼ = 0

$b_j$: estimated coefficient
$\SE(b_j)$: standard error, $\hat\sigma\sqrt{[(X^\top X)^{-1}]_{jj}}$
$p$: number of coefficients including the intercept

b_j \pm t_{1-\alpha/2,\,n-p}\,\SE(b_j)

Confidence interval for a coefficient

$t_{1-\alpha/2,\,n-p}$: t critical value with n − p df

\SE(b_1) = \frac{\hat\sigma}{\sqrt{S_{xx}}}

Slope standard error in simple regression

$\hat\sigma$: residual standard error, √(RSS/(n − 2))
$S_{xx}$: $\sum (x_i-\xbar)^2$

When to use

Deciding whether the data give evidence of a (linear, adjusted) association between a predictor and y, and how large it plausibly is.

Assumptions

Independent errors with constant variance around a correct linear mean; normal errors (less important for large n). Check residual diagnostics before trusting p-values.

How to interpret

A small p-value: the data are hard to reconcile with a zero coefficient given the other predictors in the model.
The CI is the range of coefficient values compatible with the data at the chosen level; report it alongside the p-value.
More spread in x (larger $S_{xx}$ ) and more data give a smaller SE.

Common mistakes

Saying a large p-value proves "no effect"; it means the data cannot rule zero out.
Testing the intercept when it has no meaning.
Using normal critical values instead of t with n − p df in small samples.
Running many coefficient tests and reporting only the significant ones.

Example

Spend–sales data (n = 6): $b_1 = 1.2$ , $\hat\sigma = 0.837$ , $S_{xx} = 17.5$ .

$\SE(b_1) = 0.837/\sqrt{17.5} = 0.200$ , $t = 6.0$ on 4 df, p = 0.0039.
95% CI: $1.2 \pm 2.776 \times 0.2 = (0.645, 1.755)$ sales units per thousand SAR.