The data on > that we observe can be any one of the 8 possible cases. For instance, we can have

If the error term has a continuous distribution, say a normal distribution with mean 0 and variance 1. then for each value of x we have a normal distribution for and the value of we observe can be any observation from this distribution. For instance, if the relationship between and x is

= 1 + X -

where the error term is N(0, 1), then for each value of x, will have a normal distribution. This is shown in Figure 3.1. The line we have drawn is the deter-

2 + 3c

Possible values of

for given value of x

Figure 3.1. A stochastic relationship.

ministic relationship = 2 + x. The actual values of for each x will be some points on the vertical lines shown. The relationship between and x in such cases is called a stochastic or statistical relationship.

Going back to equation (3.1), we will assume that the function f(x) is linear in X, that is,

fix) = a + px

and we will assume that this relationship is a stochastic relationship, that is,

= a + x + (3.2)

where u, which is called an error or disturbance, has a known probabiUty distribution (i.e., is a random variable).

In equation (3.2), a + px is the deterministic component of and is the stochastic or random component, a and p are called regression coefficients or regression parameters that we estimate from the data on and x.

There is no reason why the deterministic and stochastic components must be additive. But we will start our discussion with a simple model and introduce compUcations later. For this reason we have taken fix) to be a linear function and have assumed an additive error. Some simple alternative functional forms are discussed in Section 3.8.

Why should we add an error term ? What are the sources of the error term in equation (3.2)? There are three main sources:

1. Unpredictable element of randomness in human responses. For instance, if = consumption expenditure of a household and x = disposable income of the household, there is an unpredictable element of randomness in each households consumption. The household does not behave like a machine. In one month the people in the household are on a spending spree. In another month they are tightfisted.

2. Effect of a large number of omitted variables. Again in our example x is not the only variable influencing y. The family size, tastes of the family, spending habits, and so on, affect the variable y. The error is a catchall for the effects of all these variables, some of which may not even be quantifiable, and some of which may not even be identifiable. To a certain extent some of these variables are those that we refer to in source 1.

3. Measurement error in y. In our example this refers to measurement error in the household consumption. That is, we cannot measure it accurately. This argument for is somewhat difficult to justify, particularly if we say that there is no measurement error in x (household disposable income). The case where both and x are measured with error is discussed in Chapter 11. Since we have to go step by step and not introduce all the complications initially, we will accept this argument; that is, there is a measurement error in but not in x.

In summary, the sources of the error term are:

1. Unpredictable element of randomness in human response.

2. Effect of a large number of variables that have been omitted.

3. Measurement error in y.

If we teve n observations on and x, we can write equation (3.2) as

y, = a + px, + u, I = 1, 2, .... n (3.3)

Our objective is to get estimates of the unknown parameters a and in equation (3.3) given the observations on and jc. To do this we have to make some assumptions about the error terms The assumptions we make are:

1. Zero mean. ,) = 0 for all i.

2. Common variance, var ( ,) = cr for all i.

3. Independence, and «, are indepdndent for all / ¥=j.

4. Independence of Xj. u, and Xj are independent for all / and j. This assumption automatically follows if x are considered nonrandom variables. With reference to Figure 3.1, what this says is that the distribution of does not depend on the value of x.

5. Normality, u, are normally distributed for all /. In conjunction with assumption 1, 2, and 3 this implies that u, are independently and normally distributed with mean zero and a common variance cr. We write this as u, ~ IN(0, o).

These are the assumptions with which we start. We will, however, relax some of these assumptions in later chapters.

Assumption 2 is relaxed in Chapter 5. Assumption 3 is relaxed in Chapter 6. Assumption 4 is relaxed in Chapter 9.

As for the normality assumption, we retain it because we will make inferences on the basis of the normal distribution and the t and F distributions. The first assumption is also retained throughout.

Since { ) = 0 we can write (3.3) as

= a + ; (3.4)

This is also often termed the population regression function. When we substitute estimates of the parameters a and in this, we get the sample regression function.

We will discuss Hiree methods for estimating the parameters a and p.

J. The method of moments.

2. The method of least squares.

3. The method of maximum likelihood.

The first two methods are discussed in the next two sections. The last method is discussed in the appendix to tMs chapter. In the case of the simple regression