Table 7.3 Imports, Production, Stock Formation, and Consumption in France (Millions of New Francs at 1959 Prices)

Source. E. Mallinvaud, Statistical Mettiods of Econometrics, 2nd ed. (Amsterdam: North-Holland, 1970), p. 19.

The is very high and the F-ratio is highly significant but the individual t-ratios are all insignificant. This is evidence of the multicoUinearity problem. Chatterjee and Price argue that before any further analysis is made, we should look at the residuals from this equation. They find (we are omitting the residual plot here) a distinctive pattern-the residuals declining until 1960 and then rising. Chatterjee and Price argue that the difficulty with the model is that the European Common Market began operations in 1960, causing changes in import-export relationships. Hence they drop the years after 1959 and consider only the 11 years 1949-59. The regression results now are as follows:

The residual plot (not shown here) is now satisfactory (there are no systematic patterns), so we can proceed. Even though the is very high, the coefficient of x, is not significant. There is thus a multicollinearity problem.

To see what should be done about it, we first look at the simple correlations among the explanatory variables. These are = 0.026, r], = 0.99, and I23 = 0.036. We suspect that the high correlation between x, and Xj could be the source of the trouble.

Does principal component analysis help us? First, the principal components (obtained from a principal components program)"" are:

Zi = 0.7063Z, -I- 0.04352 + 0.7065 Z2 = -0.0357Z, + 0.99902 - 0.02583 Zj = -0.7070Z - 0.00702 + 0.7072

Xl, X2, Xj are the normalized values ofx,, Xj, x,. That is, AT, = (x, - ,)/( ,, X2 = (X2 - m2)/(T2, and AT, - ( - mh, where m,, mj, are the means and (T,, aj, a, are the standard deviations of x„ Xj, x,, respectively. Hence

var(A:,) = var(A2) = var(A:3) = 1

The variances of the principal components are

var(z,) = 1.999 var(z2) = 0.998 var(z3) = 0.003

Note that var(z,) = E var(Z,) = 3. The fact that var(z3) = 0 identifies that Unear function as the source of multicollinearity. In this example there is only one such linear function. In some examples there could be more. Since ,) = 2) = { = 0 because of normalization, the zs have mean zero. Thus Z3 has mean zero and its variance is also close to zero. Thus we can say that Z3 ~ 0. Looking at the coefficients of the X\, we can say that (ignoring the coefficients that are very small)

Zl = 0.706(, + Xy)

Z2X2

Z3 - 0.707(3 - Xl) Z3 - 0 gives us Xl - 3

Actually, we would have gotten the same result from a regression of Xj on AT,. The regressison coefficient is r„ = 0.9984. (Note that AT, and X are in standardized form. Hence the regression coefficient is , .)

In terms of the original (nonnormalized) variables the regression of x, on x,

X3 = 6.258 + 0.686X, = 0.998

(0.0077)

"These are from Chatterjee and Price, Regression Analysis, p. 161. The details of how the principal components are computed need not concern us here.

Of course, we can estimate a regression of X and Xj. The regression coefficient is 1.451. We now substitute for x, and estimate a regressison of on x, and x3. The results we get are slightly better (we get a higher R). The results are:

The coefficient of x3 now is (P3 + 1.451P).

We can get separate estimates of p, and only if we have some prior information. As this example as well as the example in Section 7.3 indicate, what multicoUinearity implies is that we cannot estimate individual coefficients with good precision but can estimate some linear functions of the parameters with good precision. If we want to estimate the individual parameters, we would need some prior information. We will show that the use of principal components implies the use of some prior information about the restrictions on the parameters.

Suppose that we consider regressing on the principal components z, and Z2 (Zi is omitted because it is almost zero). We saw that z, = 0.7( -I- X) and

(The figure in parentheses is the SE.)

In a way we have got no more information from the principal component analysis than from a study of the simple correlations in this example. Anyway, what is the solution now? Given that there is an almost exact relationship between X, and x„ we cannot hope to estimate the coefficients of x, and Xj separately. If the original equation is

= Po + Pll + P22 + + «

then substituting for in terms of we get

= (po -I- 6.258 ) + (Pi + 0.68 ) + px -I-

This gives the linear functions of the Ps that are estimable. They are -6.258 , p, + O.686P3, and P2. The regression of on X and Xj gave the following results.