Statistics Quest

Transformations in regression

Taking logs (or other transformations) of y or x can straighten curved relationships and stabilise variance, and changes how coefficients are read.

ln⁡y=b0+b1x  ⇒  each unit of x multiplies y by eb1\ln y = b_0 + b_1 x \;\Rightarrow\; \text{each unit of } x \text{ multiplies } y \text{ by } e^{b_1}
Log response: percentage change per unit of x is 100(e^b₁ − 1)%
b1b_1
slope on the log scale; for small |b₁|, 100·b₁ % is a good approximation
eb1e^{b_1}
multiplicative effect on (the median of) y
y=b0+b1ln⁡xy = b_0 + b_1 \ln x
Log predictor: a 1% increase in x changes y by about b₁/100
b1b_1
change in y when x is multiplied by e (≈ 2.718)
ln⁡y=b0+b1ln⁡x\ln y = b_0 + b_1 \ln x
Log–log: b₁ is an elasticity (≈ % change in y per 1% change in x)
b1b_1
elasticity

When to use

  • Right-skewed positive responses (prices, incomes, counts of sales), growth processes, residual funnels that widen with the fitted value, or curved scatter plots.
  • Logs need strictly positive values.

How to interpret

  • Log y: effects are multiplicative (percentages). Back-transforming a prediction, ey^e^{\hat y}, estimates the median of y (not the mean) when log-errors are symmetric.
  • Choose the transformation from the residual plot and subject knowledge, then re-check the residuals.

Common mistakes

  • Reading a log-scale slope as a change in units of y.
  • Comparing R2R^2 between a model for y and a model for ln y — they explain different quantities.
  • Using 100·b as the percentage when b is large (b = 0.5 means +65%, not +50%).
  • Adding an arbitrary constant before logging zeros without saying so.

Example

y = 10, 12, 15, 18, 22, 27 at x = 0…5. A straight line leaves U-shaped residuals; regressing ln y on x gives ln⁡y^=2.298+0.1991x\ln\hat y = 2.298 + 0.1991x (R2=0.9995R^2 = 0.9995, residuals patternless).

  • Each step in x multiplies y by e0.1991=1.220e^{0.1991} = 1.220: a 22.0% increase (the 100·b shortcut says 19.9%).
  • Predicted y at x = 0: e2.298=9.95e^{2.298} = 9.95.