(a) Regression of on x. (b) Regression of x on

Figure 3.4. Minimization of residual sum of squares in the regression of on x and x on y.

and X and of on y. A question arises as to which of these is the appropriate one. Following are some general guidelines on this problem.

1. If the model is such that the direction of causation is known (e.g., if we say that advertising expenditures at time-? influence sales at time t but not the other way around), we should estimate the regression using sales as the explained variable and advertising expenditures as the explanatory variable. The opposite regression does not make sense. We should estimate this equation whether our objective is to estimate sales for given advertising expenditures or to estimate advertising expenditures for given sales (i.e., always estimate a regression of an effect variable on a causal variable).

2. In problems where the direction of causation is not as clear cut, and where and x have a joint normal distribution, both the regressions on X and JC on make sense and one should estimate a regression of on : to predict given x and a regression of on to predict x given y.

3. In models where and x are both measured with error, we have to estimate both the regressions of on x and : on to get "bounds" on p. This problem is discussed in Chapter 11.

4. The case of salary discrimination mentioned earlier is an example where the problem can be posed in two different and equally meaningful ways. In such cases both regressions make sense.

5. Sometimes, which regression makes sense depends on how the data are generated. Consider the data presented in Table 3.2. : is labor-hours of work and is output, and the observations are for different workers. Which of the two regressions makes sense depends on the way the data

3.5 Statistical Inference in the Linear Regression Model

In Section 3.4 we discussed procedures for obtaining the least squares estimators. To obtain the least squares estimators of a and p we do not need to assume any particular probability distribution for the errors «,. But to get interval estimates for the parameters and to test any hypotheses about them, we need to assume that the errors u, have a normal distribution. As proved in the appendix to this chapter, the least squares estimators are

1. Unbiased.

2. Have minimum variance among the class of all linear unbiased estimators. S

The properties hold even if the errors u, do not have the normal distribution provided that the other assumptions we have made are satisfied. These assumptions, we may recall, are:

1. Eiu,) = 0.

2. V(m,) = CT for all i.

3. u, and Uj are independent for / j.

4. X, are nonstochastic.

We will now make the additional assumption that the errors u, are normally distributed and show how we can get confidence intervals for a and p and test any hypotheses about a and p. We will relegate the proofs of the propositions to the appendix and state the results here.

First, we need the sampling distributions of a and p. We can prove that they have a normal distribution (proofs are given in the appendix). Specifically, we have the following results:

were generated. If the workers are given some hours of work {x), and the output they produced (y) was observed, then a regression of on x is the correct one to look at. In this case x is the controlled variable. On the other hand, if the workers were assigned some amount of output (y) and the hours of work (x) that they took to produce that output was observed, it is a regression of x on that is meaningful. In this case is the controlled variable. Here what we are considering is an experiment where one of the variables is controlled or fixed. With experimental data which of the variables should be treated as dependent and which as independent will be clear from the way the experiment was conducted. With most economic data, however, this is not a clear-cut choice.

& mad are jointly normally distributed with

£(«) = a and

var(p) =

cov{a, ) = ff (-)

These results would be useful if the error variance were known. Unfortunately, in practice, cr- is not known, and has to be estimated. If RSS is the residual sum of squares, then

=-- is an unbiased estimator for

n - 2

Also

has a x*-distribution with degrees of freedom (n - 2)

FlJrther the distribution of RSS is independent of the distributions of a and p. (Proofs of these propositions are relegated to the appendix.)

This result can be used to get confidence intervals for cr or to test hypotheses about ct-. However, we are still left with the problem of making inferences about a and p. For this purpose we use the f-distribution.

We know that if we have two variables x, ~ N(0, 1) and X2 ~ with degrees of freedom and .x, and as independent, then

Xl standard normal

Vxjk \ independent averaged

has a r-distribution with d.f. k.

In this case (P - P)/Va/S„ ~ N(0, 1). (We have subtracted the mean and divided it by the standard deviation.) Also. RSS/a ~ xl-2 and the two distributions are independent. Hence we compute the ratio

(n - 2)tr

has a r-distribution with d.f. (n - 2). Now cr cancels out and writing RSS/ in - 2) as d- we get the result that (P - p)/Vd/5„ has a l-distribution with