This is the regression equation we would have estimated if we assumed that = ( ,/ ) , = 1.4536 ,. Thus the principal component regression amounts, in this example, to the use of the prior information = 1.4536 ,.

If all principal components are used, it is exactly equivalent to using all the original set of explanatory variables. If some principal components are omitted, this amounts to using some prior information on the ». In our example the question is whether the assumption = 1.45 makes economic sense. Without having more disaggregated data that break down imports into consumption and production goods we cannot say anything. Anyway, with 11 observations we cannot hope to answer many questions. The purpose of our analysis has been merely to show what principal component regression is and to show that it implies some prior information.

7.7 Dropping Variables

The problem with multicollinearity is essentially lack of sufficient information in the sample to permit accurate estimation of the individual parameters. In some cases it may be the case that we are not interested in all the parameters. In such cases we can get estimators for the parameters we are interested in that have smaller mean square errors than the OLS estimators, by dropping some variables. Consider the model

= 11 + 22 + « (7.15)

Zl = - We have to transform these to the original variables. We get

\ CT,

0.7 / CT, \

= - JC, + - + a constant CT, V /

Z2 = - (X2 -

Thus, using Zl as a regressor is equivalent to using X2, and using , is equivalent to using X, + ( ,/ ) . Thus the principal component regression amounts to regressing on (x, + ( ,/ ( ) and X2. In our example CTi/ctj = 1.4536. The results are:

5„(1 - r],)

[see formula (7.1)]. For the OV estimator we have to compute £(p;) and var(p;). Now

substituting for from (7.15) we get

. E JlOlJl + 22 + )

= Pi + :r P2 +

Jii -Jii

(Note that we used 5,, = x\ and 5,2 = 2 •i-2-) Hence

;) = p, + p21

( " \ Sn var(p,) = var --- 1 = •-j- = -

\ ll / " >3,,

Thus Pi is biased but has a smaller variance than p,. We have

var(p;) , ,2 var(p,) -

and if r,2 is very high, then var (pj) will be considerably less than var(p,). Now

(Bias in p;)2 / p252 var(p,) \ S,J

5,i(] - ry ct2

Sn „2 522(1 - ?2)

and the problem is that x, and are very highly correlated. Suppose that our main interest is in i- Then we drop Xi and estimate the equation

= + V (7.16)

Let the estimator of 1 from the complete model (7.15) be denoted by 1 and the estimator of 1 from the omitted variable model be denoted by 1- Pi is the OLS estimator and pj is the OV (omitted variable) estimator. For the OLS estimator we know that

,) = p, and var(p,) =

var(p2)

tl is the "true" /-ratio for Xj in equation (7.15), not the "estimated" /-ratio.

Noting that mean-square error MSE = (bias) 4- variance, and that for 0i, MSE = variance, we get

MSEQ;) (bias in fiy var(pl) MSE(p,) var(p,) var(p,)

= r\A + (1 - r],)

= 1 + rUt\ - 1) (7.17)

Thus if I/2I < 1, then MSEOJ) < MSE(P,). Since /2 is not known, what is usually done is to use the estimated /-value from equation (7.15). As an estimator of p„ we use the conditional-omitted-variable (COV) estimator, defined as

3, the OLS estimator if j/l > 1 p; the OV estimator if I/2I < 1

Also, instead of using p, or pJ, depending on /2 we can consider a linear combination of both, namely

Xp, + (1 - x)p;

This is called the weighted (WTD) estimator and it has minimum mean square error if X = /5/(1 + /). Again /2 is not known and we have to use its estimated value ij. This weighted estimator was first suggested by Huntsberger. The COV estimator was first suggested by Bancroft. Feldstein" studied the mean-squared error of these two estimators for different values of /2 and /2. He argues that:

1. Omitting a collinear nuisance variable on the basis of its sample /-statistic /2 is generally not advisable. OLS is preferable to any COV estimator unless one has a strong prior notion that I/2I is < 1.

2. The WTD estimator is generally better than the COV estimator.

3. The WTD estimator is superior to OLS for I/2I 1.25 and only sUghtly inferior for 1.5 < I/2I s 3.0.

4. The inadequacy of collinear data should not be disguised by reporting results from the omitted variable regressions. Even if a WTD estimator is used, one should report the OLS estimates and their standard errors to let readers judge the extent of multicollinearity.

"M. S. Feldstein, "Multicollinearity and the Mean Square Error of Alternative Estimators," Econometrica, Vol. 41, March 1973, pp. 337-345. References to the papers by Bancroft, Huntsberger, and others can be found in that paper.

The first term in this expression is rjj. The last term is the reciprocal of varCPj) [see formula (7.2)J. Thus the whole expression can be written as rljl where