back start next


[start] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] [55] [56] [57] [58] [59] [60] [61] [62] [63] [64] [65] [66] [67] [68] [69] [70] [71] [72] [73] [74] [75] [76] [77] [78] [79] [80] [81] [82] [83] [84] [85] [86] [87] [88] [89] [90] [91] [92] [93] [94] [95] [96] [97] [98] [99] [100] [101] [102] [ 103 ] [104] [105] [106] [107] [108] [109] [110] [111] [112] [113] [114] [115] [116] [117] [118] [119] [120] [121] [122] [123] [124] [125] [126] [127] [128] [129] [130] [131] [132] [133] [134] [135] [136] [137] [138] [139] [140] [141] [142] [143] [144] [145] [146] [147] [148] [149] [150] [151] [152] [153] [154] [155] [156] [157] [158] [159] [160] [161] [162] [163] [164] [165] [166] [167] [168] [169] [170] [171] [172] [173] [174] [175] [176] [177] [178] [179] [180] [181] [182] [183] [184] [185] [186] [187] [188] [189] [190] [191] [192] [193] [194] [195] [196] [197] [198] [199] [200] [201] [202] [203] [204] [205] [206] [207] [208] [209] [210] [211] [212]


103

Exercise

As yet another example, consider the case

XX =

5 2 17 2 3 9 17 9 60

L17 9 60J

Show that p, + , and + are estimable. Is , - + 2 estimable? Is Pi + + estimable?

The "Condition Number" Measure of MulticoUinearity {Section 7.3)

The preceding discussion referred to the case where (XX) was singular. In actual practice the variables are not exactly linearly dependent but almost are. That is, (XX) is close to singularity. The question is: How do we measure closeness? For this purpose the condition number has been suggested. It is defined as the square root of the ratio of the largest to the smallest eigenvalues (or characteristic roots) of the matrix (XX). To understand this we have to define characteristic roots.

Characteristic Roots and Vectors

Let A be an X symmetric matrix. Consider minimizing the quadratic form xAx subject to the condition xx = 1. Introducing the Lagrangian multipler X, we minimize

xAx - \(xx - I)

Differentiating with respect to x and equating the derivatives to zero, we get

2Ax - 2Xx = 0 or (A - XI)x = 0

In order that this set of equations should have a nonnull solution, we should have

Rank(A - XI) < rt or A - XI = 0

The roots of this determinantal equation, which is called the characteristic equation, are called the characteristic roots of A (alternative terms are latent roots or eigenvalues). The determinental equation A - XI = 0 is an «th-degree equation in X and has n roots. Corresponding to each solution X, there is a vector X, that is a solution of (A - X,I)x = 0. These vectors are called characteristic vectors (or latent vectors or eigenvectors). For instance, if A is 3 x 3 matrix, we have to solve the equation

a,, - X

«21

a22 - X

«33 - >

which is a cubic in X and has three roots. In the text we showed the calculation of the characteristic roots and the condition number for a 2 x 2 matrix.



As an example, consider the matrix

4 2

3 -4

Before we go to the characteristic roots of 3 x 3 matrix, let us consider the 4 2I

. The characteristic equation is:

2x2 submatrix

2 1

4 - \ 2

1 - X

or (4 - X)(l - X) - 4 = 0 5x = 0 X(X - 5) = 0. Thus, the two roots are X = 0 and X = 5. Note that the sum of the roots is equal to the sum of the diagonal elements.

For X = 0, the characteristic vector is obtained by solving the equations:

"4

"0"

4jc, + 2x2 = 0 2x, + X2 = 0

This gives Xj = -2x,. Normalizing this by taking x] + x\ = 1 or x] + 4x? = 1, we get X, = 1/V5, X2 = -2/V5- Thus, the characteristic vector is (l/\/5, -2V5). For the root X = 5, we have to solve the equations:

4-5 2

1 - 5

"0

-X + 2x2 = 0 2x, - 4x2 = 0

This gives x, = 2x2- Again normalizing using x, + x = 1 or Ax\ + xl = 1, we get X2 = 1/V5, X, = 2/V5. Hence, the characteristic vector is (2/V5, I/V5)- Thus, we have the characteristic roots and vectors as:

X = 0 vector

X = 5 vector

V5. 2 J

V5V5,

Note that the two vectors are orthogonal. Returning to the 3 x 3 matrix we have the characteristic equation

4- 2 3

1 - X -4

This gives X(X - 3)(X - 6) = 0. Thus X = 0, 3, and 6 are the characteristic roots. To get the characteristic vectors, we solve the equation (A - XI)x = 0. For X = 0 we have

2"

"0"

-Xl.



We get the equations

2x, + 2 + = 0 3X - 4x2 + 4 = 0

This gives Xj = fx, and x3 = - yx, or x, = 1, Xj = i, and x3 = - We have to normalize this vector by taking x] + xj + x] = 1. This gives the normaUzed vector as

/ 8 5

v V210

For \ = 3 we solve

"4 - 3 2 2

" "

2 1-3 1

3 - 4 4-3

. .

We get

X, + 2x2 + 2x3 = 0

2X - 2x2 + x3 = 0

3xj - 4x2 + x3 = 0

This gives x2 = ix, and x3 = -x,. Taking x, = 1 and normalizing, we get the

characteristic vector as x = (§, 3, -§). For X = 6 we proceed similarly and get

yS V6 V6

The example we have considered is that of a singular matrix (the first row is twice the second row). Hence one of the characteristic roots is zero. The matrix is also nonsymmetric. In this case the characteristic vectors are not orthogonal to each other. In the case of a symmetric matrix, they are orthogonal (as proved later). As an example, consider the symmetric matrix

9 3 3 3 1 1 3 1 7

Using the previous procedure we get the characteristic roots as X = 0, 5, and 12. The corresponding characteristic vectors are:

For X

= 0: x, = (

For X = 5:

V35 V35 V35 J



[start] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] [55] [56] [57] [58] [59] [60] [61] [62] [63] [64] [65] [66] [67] [68] [69] [70] [71] [72] [73] [74] [75] [76] [77] [78] [79] [80] [81] [82] [83] [84] [85] [86] [87] [88] [89] [90] [91] [92] [93] [94] [95] [96] [97] [98] [99] [100] [101] [102] [ 103 ] [104] [105] [106] [107] [108] [109] [110] [111] [112] [113] [114] [115] [116] [117] [118] [119] [120] [121] [122] [123] [124] [125] [126] [127] [128] [129] [130] [131] [132] [133] [134] [135] [136] [137] [138] [139] [140] [141] [142] [143] [144] [145] [146] [147] [148] [149] [150] [151] [152] [153] [154] [155] [156] [157] [158] [159] [160] [161] [162] [163] [164] [165] [166] [167] [168] [169] [170] [171] [172] [173] [174] [175] [176] [177] [178] [179] [180] [181] [182] [183] [184] [185] [186] [187] [188] [189] [190] [191] [192] [193] [194] [195] [196] [197] [198] [199] [200] [201] [202] [203] [204] [205] [206] [207] [208] [209] [210] [211] [212]