will both increase by \. Thus we get the ridge regression estimator. This interpretation makes the ridge estimator somewhat suspicious. Smith and Campbell" say that a one-liner summary of this is: "Use less precise data to get more precise estimates."

These are situations in which the ridge regression can be easily justified. In almost all other cases, there is subjective judgment involved. This subjective judgment is sometimes equated to "vague prior information." The Bayesian methods allow a systematic analysis of the data with "vague prior information" but a discussion of these methods is beyond the scope of this book.

Because of the deficiencies of ridge regression discussed above, the method is not recommended as a general solution to the multicoUinearity problem. Particularly the simplest form of the method (where a constant \ is added to each variance) is not very useful. Nevertheless, for the sake of curiosity, we will present some results on the method. For the consumption function data in Table 7.1 we estimated the regression equation

+ 2 ,-2 + • • • + 3gy,-g + U,

Needless to say, the y/s are highly intercorrelated. The results are presented in Table 7.2. Note that as X increases, there is a smoothing of the coefficients and the estimate of Po declines. The OLS coefficients, of course, are very erratic. But the estimates of Po (portion of current income going into current consumption) are implausibly low with the ridge regression method. The sudden pickup of coefficients after the fifth quarter is also something very implausible. Maybe we can just estimate the effects only up to four lags. The OLS estimates are erratic even with four lags. The computation of the ridge regression estimates with four lags is left as an exercise.

Table 7.2 Ridge Estimates for Consumption Function Data

Value of

"Gary Smith and Frank Campbell, "A Critique of Some Ridge Regression Methods" (with discussion), Journal of the American Statistical Association, Vol. 75, March 1980, pp. 74-103.

7.6 Principal Component Regression

Another solution that is often suggested for the multicoUinearity problem is the principal component regression, which is as follows. Suppose that we have explanatory variables. Then we can consider linear functions of these variables:

Z = OiX, -I- 02-2 + • • • +

Zl = byXy + biXi + • • • + bx etc.

Suppose we choose the as so that the variance of Z is maximized subject to the condition that

a] + aj + + al = I

This is called the normalization condition. (It is required or else the variance of Zl can be increased indefinitely.) Zi is then said to be the first principal component. It is the linear function of the xs that has the highest variance (subject to the normalization rule).

The detailed derivation of the principle components is given in the appendix. We will discuss the main features and uses of the method which are easy to understand without the use of matrix algebra. Further, for using the method there are computer programs available that give the principal components (zs) given any set of variables x,, X2.....X/,.

The process of maximizing the variance of the Unear function z subject to the condition that the sum of squares of the coefficients of the xs is equal to 1, produces solutions. Corresponding to these we construct k linear functions Z, Z2, • • . , Zk- These are called the principal components of the xs. They can be ordered so that

var(zi) > var(z2) > • • • > var(z/i)

Zl, the one with the highest variance is called the first principal component, Z2 with the next highest variance is called the second principal component, and so on. These principal components have the following properties:

1. var(zi) + var(z2) + + var(Zi) = var(X) -I- var(x2) + + var(xj.

2. UnUke the xs, which are correlated, the zs are orthogonal or uncorrelated. Thus there is zero multicoUinearity among the zs.

Sometimes it is suggested that instead of regressing on x,, X2, . . . , x, we should regress on Z, Z2. • - • . z,,- But this is not a solution to the multicoUinearity problem. If we regress on the zs and then substitute the values of zs in terms of xs, we finally get the same answers as before. This is similar to the example we considered in Section 7.4. The fact that the zs are uncorrelated does not mean that we wUl get better estimates of the coefficients in the original regression equation. So there is a point in using the principal components only if we regress on a subset of the zs. But there are some problems with this procedure as well. They are:

"E. Malinvaud, Statistical Methods of Econometrics, 2nd ed. (Amsterdam: North-Holland, 1970).

"S. Chatterjee and B. Price, Regression Analysis by Example (New York: Wiley. 1977).

1. The first principal component z,, although it has the highest variance, need not be the one that is most highly correlated with y. In fact, there is no necessary relationship between the order of the principal components and the degree of correlation with the dependent variable y.

2. One can think of choosing only those principal components that have high correlation with and discard the rest, but the same sort of procedure can be used with the original set of variables Xi, Xj, . . . , x,, by first choosing the variable with the highest correlation with y, then the one with the highest partial correlation, and so on. This is what "stepwise regression programs" do.

3. The linear combinations zs often do not have economic meaning. For example, what does 2(income) -I- 3(price) mean? This is one of the most important drawbacks of the method.

4. Changing the units of measurement of the xs will change the principal components. This problem can be avoided if all variables are standardized to have unit variance.

However, there are some uses for the principal component method in exploratory stages of the investigation. For instance, suppose that there are many interest rates in the model (since all are measured in the same units, there is no problem of choice of units of measurement). If the principal component analysis shows that two principal components account for 99% of the variation in the interest rates and if by looking at the coefficients, we can identify them as short-term component and long-term component, we can argue that there are only two "latent" variables that account for all variations in the interest rates. Thus the principal component method will give us some guidance as to the question: "How many independent sources of variation are there?" In addition, if we can give an economic interpretation to the principal components, this is useful.

We illustrate the method with reference to a data set from MaUnvaud." We have chosen this data set because it has been used by Chatterjee and Priceto illustrate the principal component method. We will also be using this same data set in Chapter 11 to illustrate the errors in variables methods.

The data are presented in Table 7.3. First let us estimate an import demand function. The regression of on x,, Xj, gives the following results.