a particular value of this random variable. For instance, if 6 is the population mean and g(y,, , . . . , y„) = (l/«) 2 1 = s the sample mean, is an estimator of . If = 4 in a particular sample, 4 is an estimate of .

In interval estimation we construct two functions, gi(yi, 2, , y„) and giiVu 2. • • • . ) of the sample observations and say that 6 lies between and g2 with a certain probability. In hypothesis testing we suggest a hypothesis about 6 (say, 6 = 4.0) and examine the degree of evidence in the sample in favor of the hypothesis, on the basis of which we either accept or reject the hypothesis.

In practice what we need to know is how to construct the point estimator g, the interval estimator (g,, g2), and the procedures for testing hypotheses. In the classical statistical inference all these are based on sampling distributions. Sampling distributions are probability distributions of functions of sample observations. For instance, the sample mean is a function of the sample observations and its probability distribution is called the sampling distribution of y. In classical statistical inference the properties of estimators are discussed in terms of the properties of their sampling distributions.

2.6 Properties of Estimators

There are some desirable properties of estimators that are often mentioned in the book. These are:

1. Unbiasedness.

2. Efficiency.

3. Consistency.

The first two are small-sample properties. The third is a large-sample property.

Unbiasedness

An estimator g is said to be unbiased for 6 if E(g) = 6, that is, the mean of the sampling distribution of g is equal to 6. What this says is that if we calculate g for each sample and repeat this process infinitely many times, the average of all these estimates will be equal to 6. If E{g) 6, then g is said to be biased and we refer to £(g) - 6 as the bias.

Unbiasedness is a desirable property but not at all costs. Suppose that we have two estimators g, and 2. and g, can assume values far away from 6 and yet have its mean equal to 6, whereas always ranges close to 6 but has its mean slightly away from 6. Then we might prefer g2 to gi because it has smaller variance even though it is biased. If the variance of the estimator is large, we can have some unlucky samples where our estimate is far from the true value. Thus the second property we want our estimators to have is a small variance. One criterion that is often suggested is the mean-squared error (MSE), which is defined by

Consistency

Often it is not possible to find estimators that have desirable small-sample properties such as unbiasedness and efficiency. In such cases it is customary to look at desirable properties in large samples. These are called asymptotic properties. Three such properties often mentioned are consistency, asymptotic unbiasedness, and asymptotic efficiency.

Suppose that ¸„ is the estimator of e based on a sample of size n. Then the sequence of estimators ¸„ is called a consistent sequence if for any arbitrarily small positive numbers e and 6 there is a sample size n such that

Prob [¸„ - e < e] > I - 8 for all « > «0

That is, by increasing the sample size n the estimator ¸„ can be made to lie arbitrarily close to the true value of 6 with probability arbitrarily close to 1. This statement is also written as

lim „ - e < e) = 1 and more briefly we write it as

¸ e or plim ¸„ = e

(plim is "probability limit"). ¸„ is said to converge in probability to 6. In practice we drop the subscript n on ¸„ and also drop the words "sequence of estimators" and merely say that 6 is a consistent estimator for 6.

A sufficient condition for 6 to be consistent is that the bias and variance should both tend to zero as the sample size increases. This condition is often useful to check in practice, but it should be noted that the condition is not

MSE = (bias) + variance

The MSE criterion gives equal weights to these components. Instead, we can consider a weighted average iy(bias) + (1 - 1 ) variance. Strictly speaking, instead of doing something ad hoc like this, we should specify a loss function that gives the loss in using ( ,, . . . , y„) as an estimator of 6 and choose g to minimize expected loss.

Efficiency

The property of efficiency is concerned with the variance of estimators. Obviously, it is a relative concept and we have to confine ourselves to a particular class. If g is an unbiased estimator and it has the minimum variance in the class of unbiased estimators, g is said to be an efficient estimator. We say that g is an MVUE (a minimum-variance unbiased estimator).

If we confine ourselves to linear estimators, that is, g = + 2 2 -I- • • • + c„y„, where the cs are constants which we choose so that g is unbiased and has minimum variance, g is called a BLUE (a best linear unbiased estimator).

2.6 PROPERTIES OF ESTIMATORS 25

necessary. An estimator can be consistent even if the bias does not tend to .

There are also some relations in probability limits that are useful in proving consistency. These are

1. plim (c,y, + 2 2) = c, plim y, + plim . where c, and are constants.

2. plim ( , 2) = (plim y,)(plim ).

3. plim ( ,/ 2) = (plim y,)/(plim ) provided that plim 2 5 0.

4. If plim = and g(y) is a continuous function of y, then plim g(y) ~

Other Asymptotic Properties

In addition to consistency, there are two other concepts, asymptotic unbiasedness and asymptotic efficiency, that are often used in discussions in econometrics. To explain these concepts we first have to define the concept of limiting distribution, which is as follows: If we consider a sequence of random variables . . • • • with corresponding distribution functions F,, Fj, F3, . . . , this sequence is said to converge in distribution to a random variable with distribution function F if F„(x) converges to F(x) as « oo for all continuity points ofF.

It is important to note that the moments of F are not necessarily the moments of F„. In fact, in econometrics, it is often the case that F„ does not have moments but F does.

Example

Suppose that jc is a random variable with mean , 5 0 and variance and P(x = 0) = > 0. Suppose that we have a random sample of size n: x,, 2, . . . , x„. Let x„ be the sample mean. The subscript n denotes the sample size. Define the random variable y„ = ].lx„. Then £(y„) does not exist because there is a positive probability that = 3 = • • • x„ = 0. Thus y„ = 00 if jc„ = 0. However, in this case it can be shown that / (l/x„ - \l\x) has a limiting distribution that is normal with mean 0 and variance aVp.". Thus even if the distribution of y„ does not have a mean and variance, its limiting distribution has a mean l/x and variance ( \ .*.

When we consider asymptotic distributions, we consider / times the estimator because otherwise the variance is of order Mn, which 0 as « - » 00. Thus when we compare two estimators , and 2, the variances of the asymptotic distributions both 0 as « 00. Hence in discussions of efficiency, we compare the variances of the distributions of VnT, and / . We shall now define asymptotic unbiasedness and asymptotic efficiency formally.

Asymptotic Unbiasedness

The estimator Q„, based on a sample of size n, is an asymptotically unbiased estimator of 6 if the mean of the limiting distribution of / (e„ - 6) is zero. We denote this by writing

AE(0J = e (AE = asymptotic expectations)