As mentioned in previous post, I have been working with Autoregressive and Moving Average simulations.
To test the correctness of estimations by our simulations, we employ acf(Autocorrelation) and pacf(partial autocorrelation) to our use. For different order of AR and MA, we get the varying visualizations with them, such as:

• Exponential decreasing curves.
• Damped sine waves.
• Positive and negative spikes, etc.

While analyzing and writing tests for same, I also took some time to visualize that data on ilne and bar charts to get a clearer picture:

AR(1) process

AR(1) process is the autoregressive simulation with order p = 1, i.e, with one value of phi.
Ideal AR(p) process is represented by:

To simulate this, install statsample-timeseries from here.

1
2
3
4

require 'statsample-timeseries'
include Statsample::ARIMA
series = ARIMA.new
ar_1 = series.ar_sim(1500, [0.9], 2)

Here, number of observations, n = 1500 (greater value is preferrable for best fit), p = 1, with phi = [0.9].

ACF

To generate it’s autocorrelation

1
2

acf = ar_1.to_ts.acf
p acf

For an AR(1) process, acf must exponentially decay if phi > 0, or alternate in sign if phi < 0Ref. Go through the analysis above. It can be visualized as:
When phi > 0, acf decreases exponentially:

When phi < 0, you get the alternate acf lags:

PACF

To generate it’s partial autocorrelation:

1
2

pacf = ar_1.to_ts.pacf
p pacf

For AR(1) process, pacf must have a spike at lag 1, then 0. Former spike must be positive if phi > 0, otherwise, negative spike. Have a look at the pacf series generated above. On visualizing the data:
When phi > 0, positive lag at 1 and 0(contains 1.0):

When phi < 0, negative lag at 1:

Here is the representation of ideal acf-vs-pacf for positive phi in AR(1):

AR(P) process

Simulation of AR(p) process is similar as AR(1).

1
2
3
4
5
6

series = ARIMA.new
phi_params = [0.5, 0.3]
ar_p = series.ar_sim(1500, phi_params, 2)
#acf
acf = ar_p.to_ts.acf
pacf = ar_p.to_ts.pacf

ACF

For AR(p), acf must give a damping sine wave. The pattern is greatly dependent on the value and sign of phi parameters.
When positive content in phi coefficients is more, you will get a sine wave starting from positive side, else, sine wave will start from negative side.
Notice, the damping sine wave starting from positive side here:

and negative side here..

PACF

pacf gives spike at lag 0(value = 1.0, default) and from lag 1 to lag k. The example above, features AR(2) process, for this, we must get spikes at lag 1 - 2 as:

MA(1) process

MA(1) process is the moving average simulation with order q = 1, i.e, with one value of theta.
To simulate this, use ma_sim method from Statsample::ARIMA::ARIMA

1
2
3
4
5
6

require 'statsample-timeseries'
include Statsample::ARIMA
series = ARIMA.new
ma_1 = series.ma_sim(1500, [0.9], 2)
acf = ma_1.to_ts.acf
pacf = ma_1.to_ts.pacf

For theta > 0, for MA(1), we must get a positive spike at lag 1 as:
For theta < 0, the spike at lag 1 must be in negatie direction as:

When I put these two visualizations aside each other, the visualization seems quite fit:

MA(q) process

MA(q) process. Order = q => Number of theta coefficients = q.
Ideal MA(q) process is represented by:

ACF

Similar to AR(1) simulation, it will have spikes for lag 1 - lag p as :

PACF

In pacf of MA(q) simulation, we observe exponentially decaying/damping sine wave.

ARMA(p, q) process

ARMA(p, q) is combination of autoregressive and moving average simulations.
When q = 0, the process is called as pure autoregressive process; when p = 0, the process is purely moving average.
The simulator of ARMA can be found as arma_sim in Statsample::ARIMA::ARIMA.
For ARMA(1, 1) process, here are the comparisons of the visualizations from R and this code, which just made my day :)

Quite Fit!

Cheers,
- Ankur Goel