Simple linear regression (least squares)

The least-squares line ŷ = b₀ + b₁x predicts a quantitative response from one predictor by minimising the sum of squared residuals.

b_1 = \frac{S_{xy}}{S_{xx}} = r\,\frac{s_y}{s_x},\qquad b_0 = \ybar - b_1\xbar

Least-squares slope and intercept

$b_1$: slope: estimated change in mean y per one-unit increase in x
$b_0$: intercept: predicted y when x = 0
$S_{xy},\ S_{xx}$: $\sum(x_i-\xbar)(y_i-\ybar)$ and $\sum(x_i-\xbar)^2$
$r,\ s_x,\ s_y$: correlation and sample standard deviations

\hat y_i = b_0 + b_1 x_i,\qquad e_i = y_i - \hat y_i,\qquad \text{minimise } \sum e_i^2

Fitted values and residuals

$\hat y_i$: fitted (predicted) value for observation i
$e_i$: residual: observed minus fitted

When to use

Describing and predicting a quantitative response y from one quantitative predictor x when the scatter plot looks roughly linear.

Assumptions

For description, none beyond a sensible straight-line pattern. For inference (tests, intervals) the LINE conditions: Linear mean, Independent errors, Normal errors, Equal variance — see residual diagnostics.

How to interpret

Slope in context: "each additional unit of x is associated with an average change of $b_1$ units in y."
Intercept only matters when x = 0 is meaningful and inside the data range.
The line always passes through $(\xbar, \ybar)$ and the residuals sum to 0.

Common mistakes

Extrapolating far outside the observed x range.
Reading the slope causally from observational data.
Swapping x and y: regressing x on y gives a different line (unless $|r| = 1$ ).
Fitting a line to a clearly curved pattern; check the residual plot.

Example

x = 1, 2, 3, 4, 5, 6 (spend), y = 3, 5, 4, 7, 8, 9 (sales).

$\xbar = 3.5$ , $\ybar = 6$ , $S_{xy} = 21$ , $S_{xx} = 17.5$ , so $b_1 = 1.2$ and $b_0 = 6 - 1.2 \times 3.5 = 1.8$ .
$\hat y = 1.8 + 1.2x$ : each extra thousand SAR of spend goes with 1.2 more units of sales on average. At x = 4 the prediction is 6.6.
Residuals: 0, 0.8, −1.4, 0.4, 0.2, 0 (they sum to 0).