back start next


[start] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] [55] [56] [57] [58] [59] [60] [61] [62] [63] [64] [65] [66] [67] [68] [69] [70] [71] [72] [73] [74] [75] [76] [77] [78] [79] [80] [81] [82] [83] [84] [85] [86] [87] [88] [89] [90] [91] [92] [93] [94] [95] [96] [97] [98] [99] [100] [101] [102] [103] [104] [105] [106] [107] [108] [109] [110] [111] [112] [113] [114] [115] [116] [117] [118] [119] [120] [121] [122] [123] [124] [125] [126] [127] [128] [129] [130] [131] [132] [133] [134] [135] [136] [137] [138] [139] [140] [141] [ 142 ] [143] [144] [145] [146] [147] [148] [149] [150] [151] [152] [153] [154] [155] [156] [157] [158] [159] [160] [161] [162] [163] [164] [165] [166]


142

K(b) = £((XX)~1X£)((XX)~IX,£)

= 1)

= (XX)-]XE(ee)X(XXyl.

Under the classical assumptions E(ee) = V(e) - u2l, where I is the T x T identity matrix, so

( ) = 2( . (A. 1.13)

From here it can be shown that if one takes an arbitrary linear unbiased estimator, its variance will always be greater than the OLS estimator variance given by (A. 1.13). Therefore OLS estimators are the best linear unbiased estimators, but only when the classical assumption V(e) = rj2I holds. This assumption is often referred to as spherical disturbances.

Large-sample properties of OLS estimators are easy to derive when the regressors are assumed to be non-stochastic. OLS estimators will be consistent unless the explanatory variables include a time trend. In the case that the model includes a trend the OLS estimators are asymptotically unbiased ( ) - p) and asymptotically efficient (lim V(b) - 0). Note that spherical disturbances are not necessary for consistency of OLS. In fact OLS will be consistent even when the error process is autocorrelated (assuming that the error covariance matrix has finite eigenvalues) or unconditionally heteroscedastic (§A.3).

Properties of OLS with Stochastic Regressors

So far it has been assumed that explanatory variables are non-stochastic, which is rather unrealistic. Introducing stochastic regressors complicates the analysis a little, but it is fairly simple to show that OLS is still unbiased if the regressors are uncorrelated with the errors. In fact we need to assume that the expectation of the errors, given all information embodied in the explanatory data, is zero. That is, there is no information about the errors in X, or E(e X) = 0. This implies that each column in X is uncorrelated with £.

To see that OLS is unbiased when regressors are stochastic but uncorrelated with the errors, use (A. 1.12) to calculate the expectation of b, given the data on X. Taking conditional expectations gives

£(bX) = p + (XX)-1X£(£X),

and since E(e X) = 0, ( X) = whatever the data in X. From this it is intuitively clear that the unconditional expectation of b is p.3

3The proof uses the fact that for anv random variables X and Y, E(X) = EY(E(X\Y)), so ( ) = . X)) = Ex(&) = p.



Although unbiased, OLS will no longer be efficient. However, it will still be consistent under certain regularity conditions on the regressors and assuming they are uncorrelated (in the limit) with the errors, that is,

plim[(Xe)/r] = 0.

Applying the plim operator to (A. 1.12) gives

plimb = p + plim[(XX)/r] 1plim[(X8)/r]. (A.1.14)

The data regularity condition is that plim [(XX)/T] is positive definite, so that its inverse in (A. 1.14) exists. Then plimb = p and OLS is consistent if [( )/ ] - 0. So when regressors are stochastic the OLS estimators will be inefficient, but they will be unbiased when the regressors are uncorrelated with the error process. They will also be consistent if the data satisfy a standard regularity condition.

When are OLS Estimators Normally Distributed?

OLS estimators are normally distributed under fairly general conditions. In the classical model where errors are normally distributed and the explanatory variables are non-stochastic, the normality of OLS estimators follows from (A. 1.12). This equation implies that the distribution of b is determined by that of £, and so b has a multivariate normal distribution with mean p and variance ctXX)"1.

If errors are not normal, or regressors are stochastic, or both, then it is difficult to state any finite-sample results. However, OLS estimators will be asymptotically normally distributed with mean p and variance ct2(XX) 1 under certain conditions. In particular, if the errors are not necessarily normal but they all have the same distribution (with zero mean and finite variance) and if the regressors satisfy the usual regularity conditions then the asymptotic normality of OLS estimators will follow from the central limit theorem.

A. 1.4 Estimating the Covariance Matrix of the OLS Estimators

The formula (A. 1.13) cannot be used in practice, because the variance of the error process ct2 is unknown. The error variance ct2 is estimated as follows. First calculate the OLS residual sum of squares RSS as follows:4

RSS = ee = yy - bXy. (A. 1.15)

"The formula :>a\> that the residual sum of squares is the total sum of squares, TSS - yy. minus the explained sum of squares. ESS = b X >. so called because it is the variation in Y that is explained by the model ($A.2.4i.



To obtain an unbiased estimate of 2, divide the RSS by the degrees of freedom in the model.5 Thus an unbiased estimate of 2 is

,?2 = RSS/(7/-£). (A. 1.16)

From (A. 1.13) it follows that the estimated covariance matrix of the OLS estimators is

estK(b) = (XX)". (A. 1.17)

To illustrate formula (A. 1.17), return to the simple example of the linear trend model. Here yy = 1354, b = (8.5, 2.5) and Xy = (80, 265), so by (A.1.19) the RSS is 11.5. There are 5 observations and 2 parameters, so we have 3 degrees of freedom and s2 - 11.5/3. Finally, putting

rXYY-i - ( 55/50 -15/50 \ (xx) - 15/50 5/5QJ

into (A. 1.17) gives the estimated covariance matrix:

11.5/ 55 -15\ ( 4.217 -1.150

estK(b) 15Q 15 5j- y ll5Q 0 383

In the simple linear model higher estimates of the intercept are always associated with lower estimates of the slope, so the estimate of cov(a, b) is always negative in the simple linear model. In this example it is -1.150. The estimated values of V(a) and V(b) are 4.217 and 0.383, respectively. Note that the fact that a has greater estimated variance than does b does not imply that a is a less accurate estimate than is b: the elements of the covariance matrix, and the coefficient estimates themselves, are always in units of measurement determined by those of the corresponding variables. We could just as easily model the time trend as 10, 20, 30, 40,. . . instead of 1, 2, 3, 4,. . .. Then the coefficient estimate b would be ten times as large (i.e. 25) and its estimated variance would be 100 times as large (i.e. 38.3).

Changing units of measurement should not affect the properties of the model, but parameter estimates and their variances and covariances do depend on the units of measurement. Therefore a standardized form for the coefficients should be used to investigate the accuracy and specification of a regression model. It is common to standardize parameter estimates by dividing by the square root of their estimated variance. These f-ratios are one of the most common classical hypothesis tests; they are described in §A.2.1.

5The number of degrees of freedom would be equal to the number of observations T, but one degree of freedom is lost for every statistic that needs to be calculated in order to obtain the RSS. Since we need to calculate the estimates in b to use (A. 1.15). there are T - degrees of freedom left in the model.



[start] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] [55] [56] [57] [58] [59] [60] [61] [62] [63] [64] [65] [66] [67] [68] [69] [70] [71] [72] [73] [74] [75] [76] [77] [78] [79] [80] [81] [82] [83] [84] [85] [86] [87] [88] [89] [90] [91] [92] [93] [94] [95] [96] [97] [98] [99] [100] [101] [102] [103] [104] [105] [106] [107] [108] [109] [110] [111] [112] [113] [114] [115] [116] [117] [118] [119] [120] [121] [122] [123] [124] [125] [126] [127] [128] [129] [130] [131] [132] [133] [134] [135] [136] [137] [138] [139] [140] [141] [ 142 ] [143] [144] [145] [146] [147] [148] [149] [150] [151] [152] [153] [154] [155] [156] [157] [158] [159] [160] [161] [162] [163] [164] [165] [166]