ARMA PROCESS
An Autoregressive Moving Average Process of order (p,q) is a stochastic process defined by the following equation:
Xt = µ + φ1 Xt-1 + ... + φp Xt-p + εt + ψ1 εt-1 + ... + ψq εt-q, (1)
where εt is white noise – a stationary stochastic process with zero mean and uncorrelated values at different moments of time. The dark blue part of the equation above is the autoregressive term while the light blue part is the moving average term. Coefficient µ can be set to 0 without loss of generality by considering the modified process
X’t = Xt – µ / (1 – φ1 - ... - φp).
The standard notation for Xt is ARMA(p,q). As we can see, an ARMA process is defined in discrete time. For that reason, methods related to estimation, diagnostics, forecasting and filtering of ARMA processes are considered to be part of time series analysis. Stationary ARMA processes are of biggest interest. Stationarity means an order of magnitude easier estimation and forecasting. The conditions of stationarity can be rephrased in terms of autoregressive coefficients: the real and complex roots of characteristic equation
1 - φ1 λ - ... - φp λp = 0
must lie outside the unit circle on the complex plane.
A pure autoregressive process of order p is ARMA(p,0). It is denoted with AR(p) oftentimes. A pure moving average process of order q is ARMA(0,q). It is denoted with MA(q). Both autoregressive and moving average terms in equation (1) ensure correlation of values of Xt over time. This correlation is referred to as serial correlation. Different serial correlation patterns correspond to AR and MA terms. A pure autoregressive process of order p has autocorrelation function smoothly decaying to 0 but partial autocorrelation function dropping to 0 after p-th lag. On the other hand, a pure moving average process of order q has autocorrelation function dropping to 0 after q-th lag but partial autocorrelation function smoothly decaying to 0. Naturally, an ARMA(p,q) exhibits a mixture of both effects.
Assume we are looking at a stochastic process that can be captured well within the ARMA framework. Then the optimal order (p,q) is informally researched using the observed patters in the autocorrelation and partial autocorrelation functions. The impulse response function can be of great help as well, especially if the structure of the optimal ARMA process is complex: large values of p and q, negative coefficients, etc. After several candidate models have been identified, they are estimated. The best model is then identified using 1-2 model selection criteria. Examples of model selection criteria are AIC, BIC and cross-validation.
Typically, ARMA processes are estimated using one of the variations of the maximum likelihood method. Another estimation alternative is the generalized method of moments. In the case of AR processes the method of moments degenerates into simple Yule-Walker equations. The Yule-Walker equations are linear in the model coefficients.
ARMA processes have wide-spread use in economics, finance and engineering.
ARMA REFERENCES
Greene, W. H. (2011). Econometric Analysis (7th ed). Upper Saddle River, NJ: Prentice Hall.
Hamilton, J. D. (1994). Time Series Analysis. Princeton University Press.
Brockwell, P. J., & Davis, R. A. (1991). Time Series: Theory and Methods (2nd ed). New York: Springer.
Wei, W. W. S. (1990). Time Series Analysis: Univariate and Multivariate Methods. Redwood City, CA: Addison Wesley.
Tsay, R. S. (2005). Analysis of Financial Time Series. New Jersey: Wiley-Interscience.
BACK TO THE STATISTICAL ANALYSES DIRECTORY
IMPORTANT LINKS ON THIS SITE