Covariance and Pearson correlation

Covariance and Pearson's r measure how strongly two quantitative variables move together along a straight line.

s_{xy} = \frac{1}{n-1}\sum_{i=1}^{n}(x_i-\xbar)(y_i-\ybar)

Sample covariance

r = \frac{S_{xy}}{\sqrt{S_{xx}S_{yy}}} = \frac{s_{xy}}{s_x s_y}

Pearson correlation coefficient

t = r\sqrt{\frac{n-2}{1-r^2}},\qquad df = n-2

Test of H0: ρ = 0

$t$: test statistic, referred to a t distribution with n − 2 degrees of freedom
$\rho$: population correlation

\tanh\!\left(\operatorname{atanh}(r) \pm \frac{z_{1-\alpha/2}}{\sqrt{n-3}}\right)

Fisher-z confidence interval for ρ (needs n ≥ 4)

$\operatorname{atanh}(r)$: Fisher's z transform of r, approximately normal with SE $1/\sqrt{n-3}$
$z_{1-\alpha/2}$: standard normal critical value (1.96 for 95%)

Two quantitative variables measured on the same units, and you want the direction and strength of a linear association.
Always draw the scatter plot first: r summarises a straight-line pattern and nothing else.

Pairs are independent of each other.
For the t-test and Fisher interval to be exact: (approximately) bivariate normal data. Outliers and curvature distort r badly.

Sign gives direction; $|r|$ near 1 means the points lie close to a line, near 0 means no linear trend (there may still be a curved one).
$r^2$ is the proportion of the variance in y explained by a straight-line fit on x.
r has no units and is unchanged by changing units or swapping x and y. Covariance keeps units, so its size is hard to judge on its own.
A small p-value says the data are hard to reconcile with $\rho = 0$ ; it does not say the association is strong or causal.

Reading correlation as causation: a confounder, reverse causation or coincidence can all produce r.
Concluding "no relationship" from $r \approx 0$ when the scatter plot shows a curve.
Letting one extreme point drive r, or computing r on groups mixed together (Simpson's paradox).
Comparing covariances measured in different units.

Six weeks of advertising spend x (thousand SAR) and sales y: x = 1, 2, 3, 4, 5, 6 and y = 3, 5, 4, 7, 8, 9.

$s_{xy} = 21/5 = 4.2$ ; $S_{xx} = 17.5$ , $S_{yy} = 28$ , so $r = 21/\sqrt{17.5 \times 28} = 0.949$ .
$t = 0.949\sqrt{4/(1-0.9)} = 6.0$ on 4 df, two-sided p = 0.0039.
95% Fisher interval for ρ: (0.596, 0.995) — wide, because n = 6.