ARIMA Model

The ARIMA (autoregressive integrated moving average) is a model describing time series
yt = 1,2…= {y1,y2,…}.


The full (p,d,q)(P,D,Q)s notation is given in the form of backshift polynomials. It encompasses set of simpler models:


The differencing model DIFF(d) of order d:

The seasonal differencing model DIFF(D)s of order D and seasonality lag s:

The autoregressive model AR(p) of order p:

The seasonal autoregressive model AR(P)s of order P and seasonality lag s:

The moving average model MA(q) of order q:

The seasonal moving average model MA(Q)s of order Q and seasonality lag s:

The εt is white noise (with properties: zero mean, constant variance, independence and normality):

The backshift operator B (aka the lag operator L)

The lagged value can be expressed by means of backshift operator B (sometimes it is called the lag operator L). Having time series yt=1,2…={y1,y2,…} the operator B ’s function is to refer to value lagged n times backwards:

It has standard multiplication and division properties:

It can be also used as a polynomial:

The advantage of B operator is that time series’ differencing Δ can be calculated easily:

The first order difference is for example:

And the (rarely used) second order difference (kind of “derivative”) is:

Not to be confused with seasonal differencing:

For example, having monthly time series, the annual seasonality can be expressed through seasonal differencing:

To continue the example, in combination with first differencing:

Probability of union A∪B (and how it's derived)

Given two events A and B, with their respective probabilities P(A) and P(B), the probability of their union is given by the additive rule:

Note: Only in the case that A and B are disjoint (mutually exclusive) it simplifies to:

Although it's a simple formula, the way to derive it is not so straightforward. There are actually (only) three axioms of probability, out of which everything was constructed:

 

1. axiom states that probability cannot be negative. For any event Ei (from event space F) its probability is at least 0:

2. axiom basically says that “something must happen” (the formal definition somewhat differs in source texts, but let’s stick to WolframMathworld definition). The event space F that contains N (could be also infinity) elements is defined as:

Then the axiom simply states that:

That you can read as “probability of the entire event space is 1” or alternatively “probability that at least one event happens is 1”.



3. axiom is actually the very statement about disjoint events. If any E1 and E2 events are mutually exclusive

Then probability of their union is equal to sum of their probabilities:

Or in an extended version for n mutually exclusive events:

So after we know our three axioms, let’s get back and express probability of union A and B. With aid of Venn diagram the union A∪B can be decomposed into three disjoint sets:

So then with aid of our axioms:

What to do with three or more events? Actually the union can be always decomposed into disjoint sets thanks to the mathematical inclusion-exclusion principle. It is somewhat mind boggling way of decomposing a union of sets by addition and subtraction of subset intersections. For three events A, B, C it leads to: