Stationarity and autoregressive (AR) models

An AR(p) model predicts a stationary series from a linear combination of its own last p values; differencing first turns many trending series into stationary ones.

y_t = c + \phi_1 y_{t-1} + \dots + \phi_p y_{t-p} + \varepsilon_t, \qquad \mu = \frac{c}{1 - \sum_{i=1}^p \phi_i}

AR(p) model and its long-run mean

$\phi_i$: autoregressive coefficients
$c$: constant (intercept)
$\varepsilon_t$: white-noise innovation with variance $sigma^2$
$\mu$: process mean, towards which forecasts revert

\nabla y_t = y_t - y_{t-1}, \qquad \nabla_m y_t = y_t - y_{t-m}

First and seasonal differences

$m$: seasonal period

When to use

Short-term forecasting of a series whose current value depends on recent values, after making it stationary (constant mean and variance, autocorrelation depending only on lag).
Choose $p$ from the PACF (cut-off after lag $p$ ) or by comparing out-of-sample accuracy. ARIMA adds differencing (the "I") and moving-average error terms.

Assumptions

Stationarity: for AR(1) this needs $|\phi_1| < 1$ . Trends or changing seasonality must first be removed (differencing).
Residuals behave like white noise (check their ACF).
The engine's default fit is conditional least squares with an intercept on observations $p+1, \ldots, n$ ( $\hat\sigma^2$ = SSR/( $n - p$ )); a Yule–Walker option is available. Forecasts are the recursive conditional means.

How to interpret

$\phi_1$ near 1: shocks persist for a long time; near 0: little memory. Forecasts move geometrically back towards $\mu$ .
Forecast uncertainty grows with the horizon and levels off for a stationary process.

Common mistakes

Fitting an AR model to a trending series (spurious $\phi_1 \approx 1$ ).
Over-differencing (differencing an already stationary series adds artificial negative autocorrelation).
Choosing a high order to maximise in-sample fit.

Example

Series 50, 53, 55, 54, 51, 48, 47, 49, 52, 54, 53, 50 fitted by AR(1): $\hat\phi_1 = 0.576$ , $\hat c = 21.83$ , so $\hat\mu = 51.45$ ; $\hat\sigma^2 = 4.30$ . From the last value 50 the forecasts are $21.83 + 0.576 \times 50 = 50.62$ , then 50.97 — drifting back towards 51.45.