Thus the Bayesian beta estimate is 1.193 with standard error 0.0211, not very different from the OLS beta of 1.211 with standard error 0.0216.

However, if the prior were stated with far more certainty, with a standard error of 0.01 instead of 0.1 as in Figure 8.3b, then the Bayesian beta estimate would become quite different from the OLS estimate. From (8.13) the posterior variance estimate is

((0.021586)-2 + (0.01)"2)" = 0.0000823,

so the standard error of the posterior is just 0.009. With conjugate normal distributions the standard error of the posterior will always be less than the sample or prior standard errors, reflecting the value of more information (even if it is totally subjective). However, in the first case when the prior had a standard error of 0.1, the posterior standard error was not much different from the sample standard error. This rather small reduction in the precision of the estimate reflects the fact that the prior density was not very certain, compared to the weight of the empirical evidence given by the sample data.

Now (8.13) gives the posterior mean, that is the Bayesian beta estimate:

0.0000823(1.211/(0.021586)2 +0.8/(0.01)2) = 0.872,

quite far from the OLS beta of 1.211 and much nearer to the prior beta of 0.8.

With conjugate normal distributions the standard error of the posterior will always be less than the sample or prior standard errors, reflecting the value of more information (even if it is totally subjective)

8.3.3 Confidence in Beliefs and the Effect on Bayesian Estimates

Bayesian estimates of factor sensitivities can be substantially different from OLS estimates, depending on the certainty with which prior beliefs about the value of betas are expressed. Prior beliefs could be formed as a result of historic information, such as would be the case if the beta of a stock was observed to have varied between a certain range over a period of time. Alternatively, the model developer could specify a prior density using arbitrary parameters. The only proviso for using the formulae (8.12) and (8.13) is that the prior density be normal.

The example in §8.3.2 demonstrated that if the degree of certainty in prior information is greater than in the sample information, the posterior estimates will be closer to the prior estimates. Also the standard error of the posterior will be much less than the sample standard error when prior beliefs are expressed with a high degree of certainty (note that the standard error of the posterior is much smaller in the second case, when the prior had a standard error of 0.01). Therefore if the prior beliefs are expressed with a great deal of certainty the Bayesian estimates will be close to the prior expectation and they will be viewed as having a high degree of precision. This can happen even if the information in the prior beliefs is purely subjective, so the model developer should be wary of regarding Bayesian methods as a licence to massage sample

If the prior beliefs are expressed with a great deal of certainty the Bayesian estimates will be close to the prior expectation and they will be viewed as having a high degree of precision

To express ones views in the form of a probability distribution without under- or overstating ones degree of knowledge is an art

data in order to obtain whatever results are desired. The point is that it is just as dangerous not to include information that is available as it is to overstate ones views. To express ones views in the form of a probability distribution without under- or overstating ones degree of knowledge is an art.

8.4 Remarks on Factor Model Specification Procedures

A standard econometric approach to model specification is to begin by throwing in everything that could possibly influence the dependent variable. This testing down procedure involves examining the /-ratios on all the explanatory variables (§A.2.2) and throwing out those that have insignificant t-ratios, one by one, until a satisfactory model has been achieved. The rationale for this approach is that the quality of parameter estimates will deteriorate substantially if important variables are omitted from the specification, but will be less affected if irrelevant variables are included. OLS will be biased if important explanatory variables are omitted from the model, unless the omitted variables are completely uncorrelated with the included variables. Whether the bias on a coefficient is upward or downward depends on whether the variable has positive or negative correlation with an important but excluded variable.

However, leaving irrelevant variables in the model does not cause a problem for the estimates of parameters on the relevant variables. They will still be unbiased, and the only effect will be that the precision of all estimates will be depressed since degrees of freedom are lost as more variables are included. This causes no problems in reasonably large data sets, of course, where very many degrees of freedom are available. But one should be very wary of extreme movements in irrelevant variables. During normal times the irrelevant variables will have little effect on predictions, since their coefficients will be small. But if a large idiosyncratic movement occurs in one of these variables, the model predictions will be knocked off course. This is one of the reasons why models have to be thoroughly backtested.

It can be that two variables both appear to have little effect in the model, until one of them is removed and then the other becomes very significant

Of course there are problems with the testing down approach, particularly if data are missing, but it is a framework for model development that is commonly considered. The main problem is caused by multicollinearity (§A.4.1). When there is a high degree of multicollinearity it becomes very difficult to distinguish what is and what is not a relevant variable for the model. If some of the variables that are initially thrown in with the kitchen sink are highly collinear, this will depress the precision of parameter estimates. Thus some useful variables may appear to have little effect in the model. It can be that two variables both appear to have little effect in the model, until one of them is removed and then the other becomes very significant. Alternatively, a variable that seemed very useful suddenly may become insignificant when another (related) variable is taken out of the model.

Multicollinearity is a common problem with multi-factor models: a factor that seems to be an important determinant of returns may suddenly appear insignificant when another collinear factor is included, and factor sensitivity estimates may lack robustness as different model specifications are tested. For this reason many firms are now favouring a more objective framework, where stock returns are modelled by statistical factors alone. These models do not contradict the APT. Different APT models assume that different factors are the driving forces, but statistical factor models rely on the data alone to tell the story. The economic principles are not denied but, given the difficulty in defining a single true factor model, let alone the difficulties in estimation, more robust results may be obtained by using a statistical analysis from beginning to end.

The testing down procedure can be a little haphazard, leaving much to the integrity of the model builder. However, that is always the case with the development of fundamental models. Financial and economic theory are a great guide to model specification, but nevertheless the developer will be faced with many decisions about the variables to use and the data on these variables. Model development is as much an art as a science: it is a rare and valuable quality to be able to use a model to its greatest potential. Just as a musician who performs well on an instrument is appreciated, perhaps above the technician who built the instrument, so a practitioner who can play a model really well may be valued more than the quantitative analyst who understands how to build the model but cannot play it!

Model development is as much an art as a science: a practitioner who can play a model really well may be valued more than the quantitative analyst who understands how to build the model but cannot play it

A few words of warning are appropriate before finishing this chapter. A linear model is only a very basic formulation of a functional relationship. It is possible to capture a certain amount of non-linearity in relationships by transforming variables (e.g. with exponential or logarithmic transformations), but in the end a linear model may just be an inadequate description of the multivariate data generation process. In fact there may not be any sort of stable relationship between variables, let alone a linear one. This will show itself in the residual analysis, where no amount of inclusion of new variables or transformation of existing variables removes econometric problems such as autocorrelation, heteroscedasticity or structural breaks.

In this scenario the typical econometricians response is to employ a more advanced method of estimation. Instrumental variables are a favourite with economists because, whatever the explanatory variables in the actual model, if they are instrumented in a creative manner, one might obtain estimators that do indeed take values that are in line with whatever economic theory the researcher seeks to uphold - and these estimators will normally be consistent (§A.1.3). This is just one example of a problem that is endemic in statistical analysis: advanced techniques are sometimes regarded as methods for establishing relationships that would otherwise be obscured, rather than more sophisticated tools for refining models of relationships that are already established. But if one has to take a sledgehammer to crack a nut, one wonders

Advanced techniques are sometimes regarded as methods for establishing relationships that would otherwise be obscured. But if there is a real, stable relationship between the underlying variables it should shine through, whatever the estimation methods employed