back start next


[start] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47] [ 48 ] [49] [50] [51] [52] [53] [54] [55] [56] [57] [58] [59] [60] [61] [62] [63] [64] [65] [66] [67] [68] [69] [70] [71] [72] [73] [74] [75] [76] [77] [78] [79] [80] [81] [82] [83] [84] [85] [86] [87] [88] [89] [90] [91] [92] [93] [94] [95] [96] [97] [98] [99] [100] [101] [102] [103] [104] [105] [106] [107] [108] [109] [110] [111] [112] [113] [114] [115] [116] [117] [118] [119] [120] [121] [122] [123] [124] [125] [126] [127] [128] [129] [130] [131] [132] [133] [134] [135] [136] [137] [138] [139] [140] [141] [142] [143] [144] [145] [146] [147] [148] [149] [150] [151] [152] [153] [154] [155] [156] [157] [158] [159] [160] [161] [162] [163] [164] [165] [166] [167] [168] [169] [170] [171] [172] [173] [174] [175] [176] [177] [178] [179] [180] [181] [182] [183] [184] [185] [186] [187] [188] [189] [190] [191] [192] [193] [194] [195] [196] [197] [198] [199] [200] [201] [202] [203] [204] [205] [206] [207] [208] [209] [210] [211] [212]


48

«,,.123 - „

Note that RSS = Syy{\ - R) in all cases.

One other important relationship to note, which we have also mentioned in Chapter 3, is that if we consider the residual , we note that

, = y, - a - p,x„ - 02*2,

Thus the normal equations (4.2)-(4.4) imply that

2 , = 0 2 „ = 0 E 2, = 0 (4.10)

These equations imply that cov(u, :,) = 0 and cov{u, X2) = 0. Thus the sum of the residuals is equal to zero and the residuals are uncorrelated with both x, and X2.

Illustrative Example

With many computer packages readily available for multiple regression analysis one does not need to go through the details of the calculations involved. However, it is instructive to see what computations are done by the computer programs. The following simple example illustrates the computations.

In Table 4.1 we present data on a sample of five persons randomly drawn from a large firm giving their annual salaries, years of education, and years of experience with the firm they are working for.

Y = annual salary (thousands of dollars)

A", = years of education past high school

X2 = years of experience with the firm

The means are Y = 30, Jf, = 5, Xj = . The sums of squares of deviations from the respective means are

5„ = 16 5,2 = 12 5„ = 62 522 = 10 2 = 52

Syy = 272

The normal equations are

We can solve these equations by successive elimination. But this is cumbersome. There are computer programs now available that compute all these things once the basic data are fed in. Thus we will concentrate more on the interpretation of the coefficients rather than on solving the normal equations. Again,

RSS = Sy, - p,5„ - 2S2y - ,



Experience

Y X,

Y - Y

X, - X,

-

30 4

20 3

36 6

24 4

40 8

16p, + 1202 = 62 120, + 1002 = 52

Solving these equations, we get

0, = -0.25 02 = 5.5

a = F - 0,Z, - 022 = 30 - (-1.25) - 55 = - 23.75

= = ? . 0.998

Syy 272

Thus the regression equation is

t = -23.75 - 0.25 :, + 5.5A2 R = 0.998

This equation suggests that years of experience with the firm is far more important than years of education (which actually has a negative sign). The equation says that we can predict that one more year of experience, after allowing for years of education (or holding it constant) results in an annual increase in salary of $5500. That is, if we consider the persons with the same level of education, the one with one more year of experience can be expected to have a higher salary of $5500. Similarly, if we consider two people with the same experience, the one with an education of one more year can be expected to have a lower annual salary of $250. Of course, all these numbers are subject to some uncertainty, which we will be discussing in Section 4.3. It will then be clear that we should be dropping the variable X completely.

What about the interpretation of the constant term -23.75? Clearly, that is the salary one would get with no experience and only high school education. But a negative salary is not possible. What of the case when X2 = 0, that is, a person just joined the firm? Again, the equation predicts a negative salary! So what is wrong?

What we have to conclude is that the sample we have is not a truly representative sample from all the people working in the firm. The sample must have been drawn from a subgroup. We have persons with experience ranging from 8 to 12 years in the firm. So we cannot extrapolate the results too far out of this sample range. We cannot use the equation to predict what a new entrant would earn.

Table 4.1 Data on Salaries, Years of Education, and Years of



Y = -22.0 + 5.22 = 0.994

The simple regression equation predicts that an increase of one year of education results in an increase of $3875 in annual salary. However, after allowing for the effect of years of experience we find from the multiple regression equation that it does not result in any increase in salary. Thus omission of the variable "years of experience" gives us wrong conclusions about the effect of years of education on salaries.

4.3 Statistical Inference in the Multiple Regression Model

Again we will consider the results for a model with two explanatory variables first and then the general model. If we assume that the errors u, are normally distributed, this, together with the other assumptions we have made, impUes that u, IN(0, CT), the following results can be derived. (Proofs are similar to those in Chapter 3 and are omitted.)

1. a. Pi, and 02 have normal distributions with means a, p,, P2, respectively.

2. If we denote the correlation coefficient between :, and X2 by r,2, then var(P,)

var(P2)

cov(p„ P2)

5„(1 - 2)

522(1 - ri2)

5,2(1 - rid

var(a) = - + x] var(p,) + 2x,X2 cov(p,, Pj) + jc var(P2) n

cov(a. Pi) = -U, var(P,) + x, cov(p„ P2)] cov(a, P2) = -[xi cov(p„ P2) + 2 var(P2)]

Comments

1. Note that the higher the value of r,2 (other things staying the same), the higher the variances of p, and Pj. If r,2 is very high, we cannot estimate P, and P2 with much precision.

It would be interesting to see what the simple regressions in this example give us. We get

Y = 10.625 + 3.875A, i? = 0.883



[start] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47] [ 48 ] [49] [50] [51] [52] [53] [54] [55] [56] [57] [58] [59] [60] [61] [62] [63] [64] [65] [66] [67] [68] [69] [70] [71] [72] [73] [74] [75] [76] [77] [78] [79] [80] [81] [82] [83] [84] [85] [86] [87] [88] [89] [90] [91] [92] [93] [94] [95] [96] [97] [98] [99] [100] [101] [102] [103] [104] [105] [106] [107] [108] [109] [110] [111] [112] [113] [114] [115] [116] [117] [118] [119] [120] [121] [122] [123] [124] [125] [126] [127] [128] [129] [130] [131] [132] [133] [134] [135] [136] [137] [138] [139] [140] [141] [142] [143] [144] [145] [146] [147] [148] [149] [150] [151] [152] [153] [154] [155] [156] [157] [158] [159] [160] [161] [162] [163] [164] [165] [166] [167] [168] [169] [170] [171] [172] [173] [174] [175] [176] [177] [178] [179] [180] [181] [182] [183] [184] [185] [186] [187] [188] [189] [190] [191] [192] [193] [194] [195] [196] [197] [198] [199] [200] [201] [202] [203] [204] [205] [206] [207] [208] [209] [210] [211] [212]