COMPARING THE RESULTS OF TWO SYSTEMS

The easiest way to understand the use of testing is by exanple. Using daily IMM Swiss franc data fran Januarj 1986 through Februarj 1996, we compare the results of two trend sjstems, one based on a moving average and the other on exponential smoothing. These calculations can be found in Chapter 4 ("Trend Calculations"). In both cases, the sjstem generates a buy signal or a sell signal when the frendline tums up or down, respectively Table 21-1 shows the results of varjing the calculation interval fran 10 to 250 dajs in stqas of 10 dajs. Parts a and b give the results using a commission of $25 per trade, and parts and d use $ 100 per trade

Which Sjstem Is Better?

If we scan the results of the net profit (NetPrft) in column two of parts a and b, we can find that the largesl profit was $52,125 for the exponential smoothing at 120 days, the largest rdum on account (ROA) was 376°o for exponential smoothing at 160 dajs, and the highest profit factor was 3.47 for the moving average at 250 dajs. However, looking at intKvidual values can be contusing and misleading. Choosing the one best performer is only sensible if you can guarantee that those parameters will be the ones that perform best in the next period-an impossible request.

At the bottom of each part of Table 2 1-1 are the average and standard deviation of each set of tests. For parts a and b, the average net profits for the moving average sjstem were slightly better than exponential smoothing; however, looking at all the averages and variance of the two sjstems show that the results are nearly identical. From a practical point of view there is really no difference between the performance of the two sjstems. One might profit from one particular situation in one sjstem, but over the long run they will be esrtremely close. This may be easier to see in Figure 21-12, which shows ttie net profits of the two methods on the same chart, it may seem that the moving average technique performs slightly better with shorter intervals (30 to 100 dajs) and the exponential at longer ones (120 to 170 dajs), but these (Kstinctions are very small. The overall picture is that a trending sjstem produces profits based on the time interval used in the calculation, and is not dependent upon the apecffic method of calculation. This observation allows us to view the robustness of trend following as a trading approach.

A comparison of optimization results usually selects the one with the highest profits, but that may not be the best choice. Comparisons should account for the following minimum criteria:

1. A good sample of parameter combinations were tested. This will include those variables that cause frequent, as well as slow, trading. The (Ksfribution of fast and slow test results should be similar, which may involve scaling the calculation periods in different ways. One simple way of knowing if you have succeeded in this (Kstribution is by observing the average number of frades per test on each sjstem.

2. The average results of all tests and the variance of results should be compared. Significantly higher and more consistent overall results is a sfrong argument for a fundamentally sound a.

TABLE21-1 Comparison of Swiss Franc Trend Tests Swiss Franc 1986-1995,10 years

250 240 230 220 210 200

37363 IIOI3 26150 20963 27400

9300 18600 23088 45388 45863 39338 38313 36113 52125 12250 35800 30163 29875

7725 37538 26563 37538 30813 23938 -8975 27770 13709

1-57 1.17 1.42 I 56

3-06 2.42 1.89 1 2

0.96 1.58 0.51

-11038 28763 -I53B8 21163 -30725 -39988 33863 -25238 -13500 -12188 -21788 -22063 20163 19138 -42050 -14088 -25788 15000 31588 -30363 -20788 -19400 -26600 29638 44325 -24585 9056

TABLE 2 1 -1 (Continued)

(c}Mv«iAk , $100

FIGURE 21-12 Comparison of Swiss fcinc traid-followiiig tediniques.