Stationarity in time series

Formally: A time series y1,y2,... is nonstationary if, for some m, the joint probability distribution of yi, yi+1, ..., yi+m-1 is dependent on the time index i (definition by WolframMathworld)


Strong form of stationarity: A time series whose joint probability distribution does not change in time


Weak form of stationarity: A time series whose mean and autocovariance do not change in time (i.e. mean is the same at any time and covariance only depends on lag, not on time).


Practically: If the time series appear mean-reverting it is most probably stationary. If, on the contrary, it is drifting up or down, it is probably not. If the autocorrelation function drops to zero quickly on higher lags, it is probably stationary. On the contrary, ACF for non-stationary time series decreases very slowly. "Time series with trends, or with seasonality, are not stationary. A time series with cyclic behavior is stationary - that is because the cycles are not of fixed length, so before we observe the series we cannot be sure where the peaks and troughs of the cycles will be" (Rob J. Hydman: Forecasting: principles and practice).


The problem with non-stationarity is also known as “spurious regression” problem. When dealing with two (or more) non-stationary time series, very high correlation is often found between them, despite lack of any rational causality. What is often overlooked and ignored in many studies is that the correlation and regression coefficients might be misleading if assumptions (non-stationarity) are not met. The problem is that sample estimates of correlation and regression coefficients can not be interpreted as population parameters at all in such case (Johansen: Correlation, regression, and cointegration of nonstationary economic time series).


Treatment of non-stationarity is possible:

• If there is a deterministic trend, the time series can be in fact “trend stationary process” as it mean-reverts to a level determined for each time t by function f(t) (e.g. linear time trend f(t)=βt). By simply subtracting trend f(t) we obtain stationary process.
• If the series drifts up or down (the process is not mean-reverting because stochastic shocks persists a.k.a. there is a unit root) then differencing helps. Typical example would be random walk process.
• If there is deterministic seasonality, set of dummy variables can capture the seasonal component.
• If there is changing seasonal effect, then seasonal differencing helps.
• If there is increasing variance, log transformation can be applied (to remedy the variance problem in general, the Box-Cox transformation function helps).

The ultimate goal of non-stationarity remedies should be however to obtain stationary residuals. A model with non-stationary dependent or independent variables should not present a problem as long as model errors are stationary. A prime example would be cointegration model, which is a correct model describing two processes which evolve in an equilibristic relationship.

ARIMA prototypes

There are some interesting prototypic ARIMA models. In fact the ARIMA framework is that much general that it covers whole list of time series models:


ARIMA(0,0,0) is white noise (with optional level based on c):

ARIMA(0,1,0) is random walk (with opt. drift based on c):

ARIMA(1,1,0) is differenced first-order autoregressive model (with opt. drift based on c):

ARIMA(0,1,1) is exponential smoothing (with opt. drift based on c):

ARIMA(0,2,2) is linear exponential smoothing (with opt. curvature based on c):

ARIMA(1,1,2) is damped-trend linear exponential smoothing (with opt. drift based on c):

ARIMA(0,0,0)(1,1,0)12 is annually differenced first-order autoregressive model (with opt. drift based on c):

ARIMA(0,1,0)(0,1,0)12 is annual random trend model (with opt. curvature based on c):

It is also interesting to show that AR and MA terms can mimic differencing (more on this in Robert Nau: Statistical forecasting: notes on regression and time series analysis). It's easiest to explain on ARIMA(1,1,1) model:

It is evident, that if ϕ→1 it mimics the differencing (1-B) and, on the contrary, if θ→1 it mimics integrating, as the (1-B) on both sides of equation “cancel each” other. In a similar way if ϕ≈θ the (1- ϕB) “cancels” (1- θB) out.



It is also shown that a MA model can be expressed as infinite AR model (and vice versa). Consider simple MA(1):

Then with the “hack” of infinite geometric series expression:

Possible lesson from the “mimicking” topic would be: do not “overfit” by trying model where simultaneously both (AR and MA) term orders are 2 or more (e.g. ARIMA(2,1,2)) because the terns might cancel each other out.