to compute the exact maximum likeUhood estimates as well. An algorithm for this is described in Ansley.*

Residuals from the ARiViA Models

After obtaining estimates of the parameters (a,, aj, P,, P2), we get the predicted residuals from equation (13.6). We have

, = 1, - 1 - a2(-2

Note that t, are obtained from the As and the final estimates p, and P2 of p, and P2, respectively. These residuals are useful for forecasting from ARMA models. This is discussed in the next section.

An alternative way of obtaining the residuals is to solve the ARMA(2, 2) model by expanding the expressions in terms of the lag operator L. Note that the model (1 - a,L - a2L)x, = (1 + p.L + fiiL), gives

e, = (1 + p,L + fijlh-W - - «2

Since this is a power series in L, we should write it as:

(1 + y,L + + ? + y,L + )x,

We have to find the y,. This can be done by noting that (1 + p,L + PjL) (1 + 7,L + 72 + + •••) = (!- a,L - aV) and equating the coefficients of Uke powers of L. This gives

coefficient of L = Pi + 7i = -«1 or 7, = -(a, + p,) coefficient of = P2 + Pi7i + 72 = ~«2 or

72 = -(«2 + P2) + Pl(«l + Pi) coefficient of L? = p,72 + P271 + 73 = 0 or

73 = -(P.72 + P271)

The rest of the 7s can be obtained recursively from the relation

coefficient of L-" = 7,+, = -(Pi7j + P27,-i)

Once we get the 7,, we can write e, = :, + \ , , + 72-,-2 + • and get the estimated residual e, by substituting 7 for 7. We shall not provide examples here; they are provided in the exercises at the end of the chapter.

Testing Goodness of Fit

When an AR, MA, or ARMA model has been fitted to a given time series, it is advisable to check that the model does really give an adequate description of the data. There are two criteria often used that reflect the closeness of fit and

•C. F. Ansley, "An Algorithm for the Exact Likelihood of Mixed Autoregressive Moving Average Process," Biometrika, Vol. 66, 1979, pp. 59-65.

the number of parameters estimated. One is the Akaike information criterion (AIC), and the other is Schwartz Bayesian criterion (SBC). The latter is also called the Bayesian information criterion (BIC). If p is the total number of parameters estimated, we have

AIC(p) = n log al -I- 2p BIC(p) = log CT -t- p log n

Here n is the sample size. If RSS is the residual sum of squares, ], then CTp = RSS/(n - p). If we are considering several ARMA models, we choose the one with the lowest AIC or BIC. (The two criteria can lead to different conclusions.) These goodness-of-fit criteria are more like the or minimum CT-type criterion. In addition, we have to check the serial correlation pattern of the residuals-thai is, we need to be sure that there is no serial correlation. One can look at the first-order autocorrelation among the residuals. However, as discussed in Chapter 6, one cannot use the Durbin-Watson statistic. With autoregressive models, we have to use Durbins / -test, or the LM test discussed in Section 6.8.

Box and Pierce suggest looking at not just the first-order autocorrelation but autocorrelations of all orders of the residuals. They suggest calculating Q = N I where is the autocorrelation of lag and TV is the number of observations in the series. If the model fitted is appropriate, they argue that Q has an asymptotic distribution with m - p- q degrees of freedom, where p and q are, respectively, the orders of the AR and MA components.

Actually, though the g-statistics are quite widely used by those using time-series programs (there is no need to list here the hundreds of papers, books and programs that still use them); they are not appropriate in autoregressive models (or modelswith lagged dependent variables). The arguments against their use are exactly the same as those against the use of the DW statistic, as discussed in Chapter 6. One can use Durbins / -test but that tests for only first order autocorrelation. The g-statistics are designed to test correlations of higher orders as well. For this purpose it is appropriate to use the LM test as suggested in Secton 6.8 of Chapter 6.

The discussion in the time-series Uterature does not pay any attention to this aspect of the inappropriateness of the g-statistics. The Box-Pierce paper appeared in 1970 and Durbins paper, which showed the inappropriateness of using the DW test with lagged dependent variables (autoregressive models) and suggested an alternative, also appeared in the same year. In spite of the fact that the discussion of the g-statistics in the time-series literature was all in the 1970s after Durbins paper appeared, it all concentrated on the "low power" of the g-statistics. For instance, Chatfield and Prothero fitted four different

O. E. P. Box and D. A. Pierce, "Distribution of Residual Autocorrelations in Autoregressive Integrated Moving Average Time Series Models," Journal ofttie American Statistical Association. Vol. 65. 1970, pp. 1509-1526.

*C. Chatfield and D. L. Prothero, "Box-Jenkins Seasonal Forecasting: Problems in a Case Study." Journal ofttie Royal Statistical Society, Series A, Vol. 136, 1973, pp. 295-336.

G. M. Ljung and G. E. P. Box, "On a Measure of Lack of Fit in Time Series Models," Biometrika, Vol. 65, 1978, pp. 297-303.

"N. Davies and P. Newbold, "Some Power Studies of a Portmanteau Test of Time Series Model Specification," Biometrika, Vol. 66, 1979. pp. 153-155.

"See J. R. M. Hosking, "Lagrange Multiplier Tests of Time Series Models," Journal of the Royal Statistical Society, Series B, Vol. 42, 1980, pp. 170-181, for a general discussion. -L. G. Godfrey, "Testing the Adequacy of a Time Series Model," Biometrika, Vol. 66, 1979, pp. 67-72.

ARIMA models to the same set of data, but all four models gave a nonsignificant value of Q. Ljung and Box suggest a modification of the g-statistic for

moderate sample sizes. They suggest g* = N(N + 2) Y ~ A)"r instead

of the 2-statistic. However, even this statistic was found to have low power by Davies and Newbold." This too is inappropriate, just the way Q is, in autoregressive models.

Thus one has to replace the g-test with some other tests. One alternative that has been suggested is to use Lagrange-MultipUer (LM) tests." We discussed this earlier in Section 6.8 of Chapter 6. They are, as shown there, very simple to implement. Godfrey discusses a different type of LM test. This is to check the adequacy of the degree of auto regression in the ARMA model.

Suppose that we estimate an ARMA(p, q) model and ask whether we should be estimating the extended ARMA (p + m, q) model. As before, a, represent the parameters in the AR part and p, represent the parameters in the MA part.

Denote by e, the computed residuals from the ARMA(p, q) model. Now consider the extended ARMA(p, + m, q) model and compute

, . de, i = \,2, . . . , p + m

- = v., and - = ,.= 1,2,...,,

Let v„ and w,, be the values of v„ and w,„ respectively, evaluated at the ML estimates of the restricted ARMA(p, q) model (i.e., setting a, = 0 for / = p + \, . . . , p + m). Now estimate a regression of e, on the (p + m + q) variables v„ and vv,,. Let R be the coefficient of determination and N the number of observations. The LM statistic is given by

LM = NR

which has (asymptotically) a Xm-distribution.

Godfrey studies the finite sample properties of the LM test in the context of overfitting an AR(I) process by higher-order autoregressive models. He finds that the power of the LM tests is higher than that of the Q and Q* tests unless the number of overfitted parameters is large. In the special case where m is large, the LM test and the tests like Q and Q* coincide. This explains why the g-statistic or g*-statistic is not very useful, because when we are testing the adequacy of the ARMA(p, q) model, the alternative we have in mind is an ARMA(p + m, q) model with large m.

Godfreys procedure of using auxiliary regression of e, on v„ and Wj, is also