compared with six contracts tested during the middle period, failing off to two contracts at the end of 1996.

2. Selected parts of contracts. Segments can be selected that include the last 30 to 90 dajs of a contract. Bj-limiting a test to the period you are planning to trade, you have a good representation of expected performance, although there are a lot of litde pieces to evaluate. \Vhen operating this way, you must remember to allow a windup period for your trend calculations prior to when you begin taking trading signals.

FIGURE21-1 Testing using full conhacts from December 1992 through December 1996. (a) Test intervals, (b) Dish-ibution of testing.

Test intervals

1991 1992 1993 1994 199S 1996

Distribution Qf testing (to)

3. Badvard (or forward) adjusted data. The data segments in item 2 are combined on a apecific roll date by adjusting all the prices in that one confract by the difference between the closing prices of the new and old contract on the roll date. In a backward adjustment, all data prior to the adjustment date are changed by the differential on the roll date. This technique has become popular and wori wen for frend sjstems. The adjusbnents will cause large distortions in the oldest data, especially if the data go back more than 10 years. In the exfreme case, the acjusted price can even become negative. If the strategj uses a percentage value, this series will not give the right numbers; or, if you expect to compare various high and low prices over time, it is best to use another tjpe of series.

4. Indexed data series. A continuous series is consfructed by adding the price change each day to the prior index value. When there is a change of contract, the price change is that of the new contract compared with its prior price. The conversion into an indexed series is useful for comparing two martlets and for con

sfructmg portfolios, but has most of the same problems as its raw 1 in item 3.

5. Constant forward series. In the same manner as the London metals contracts are quoted (n-dajs out), i

artificial price is calculated by inte olating the prices of the delivery months on both sides of the date needed for this price series. This tjpe of data series has also been called a perpetual contract and presents its own set of problems. Deferred contracts are not as volatile as the nearby contract, and formulated values create a series thai is much smoother than the one you are likely to trade. This means less ride probably less profits, and greater reliability in identifying trends. More important, you would be testing your ideas on a data series in which none of the prices actually occurred.

if all five methods have problems, which is best? If you use a trend calculation and do not rely on a percentage or compare the absolute levels of past prices, then you can use backward adjusted data (item 3). Although it requires more work, the safest choice is ahvajs to use the individual contracts, provided you take care not to duplicate performance over the same dates in different contracts unless you plan to trade ttiat way.

In a seasonal or cyclic commodity, there is good reason for selecting more than one delivery month during each year-new and old crops or production cycles can cause substantially unique price pattems during ttie same time period for different deferred deliveries. Even the proper overlapping of a traditionally nonseasonal product may pick up deviations due to anticipation or extreme short-term demand or su lus. But the market shown in Figure 2I-Ia was not seasonal or cyclic, and the 18-month contract duration caused the overlapping time intervals that simply distorted the test results.

SEARCHING FOR THE OPTIMAL RESULT

There are many complex techniques for optimizing; the two most basic will be discussed here. The first, and the only simple method, is to test all combinations of parameters. A single moving average sjstem with a stoploss might have the following tests:

1. All moving average dsjs from I to 50

2. Stop-loss values from 0 to $ 1,000 (or equivalent in points), in steps of $ 10

The total number of tests would ttien be 50 x 100 = 5,000. When ttiree parameters are tested, ttie nuniber of tests increase rapidly. Taking care to use the best parameter disfribution, as discussed earlier, is a simple way to reduce the test time and improve the usefulness of the results. When there are too many tests, even with selected values, a wider apacing of values can be used to locate the general pattem of performance. Once the approximate range of parameter values is known, more detailed retesting can be done.

Calculate the number of tests you will run before beginning an optimization. Although the time may not be a problem, your test software may not be set up to hold the number of test results that you are creating. All computers have limits. If you generate as many as I combinations, remember that you will need to evaluate them using a apreadsheet. Large numbers of rows and columns, combined with numerous markets, can create an umnanageably large

Sequential Testing

If you have created a sjstem on a grand scale and the number of tests and the number of parameters are too great, then mathematical optimization offers some solutions in the form of testing procedures. One caveat is that they only apply to continuous results, that is, performance thai increases and decreases somewhat smoothly based on parameter val

ues. You cannot use this method to test coded variables that cause an abrupt shift in the methodologj. In the simplest example, the procedure is:

1. Test one parameter, such as the movmg average period.

2. Find the best result, and fix that value for the remaining tests.

3. Select the next parameter and test.

4. Go back to step 2

This IS called a sequential optimization. It has the advantage of reducmg a 3parameter test case from n, x n2 x n3 total tests to n, + n2 + n3. It has die disadvantage, however, of not alwajs finding the best combination of parameter values when the test is very laige. Figure 21-2 shows a 2dimensional map of test results, a good way of visualizing the relationships between parameter sets. By testing all the combinations, the best choice was seen to be a 20-dsy moving average with a 30point stop-loss, netting 400. Using sequential testing, beginning with the moving average dsjs, the best result is 15 dsjs. Fixing that value and testing the stop-loss values, we find that the best stoploss is at 30 points. While this result is good, it is not the peak result nor the next best 350. If the tests begin with the stop-loss value, il ends at the same point; there, the best results cannot be found, whether the test begins with the rows or columns. Even though the area of greatest profitability is clear, the best choices in the first row and column do not peak at values consistent with the overall best result.

There are two wajs that traditionally avoid this problem. One method is to list the parameters in order of importance, that is, those that would have the greatest impact on profitability, although you might choose this by its fundamental importance to the strategj. In this example, the number of dsjs in the moving average is most likely to be the primaij parameter. This doesnt work in Figure 21-2; however, mathematicians are prepared for this problem.

The final altemative is to choose a series of random starting values for each parameter and proceed to test a row or column beginning with those values. The object is to find the best results with as few of these tests as possible, Although each 3-parameter test is only n, + n2 + n3, the total number of tests will accumulate steadily. When you have performed a number of searches beginning with a random seed, and found the same results,

FIGURE 21-2 Visualizing the results of a 2-dimensional optimization. Movinsaveragedays

10 100 150 200 150 150

you can conclude with some certainty that you have located the peak. For example, if you begin searching in Figure 21-2 in four places, one row and one column inside the four comers (moving average of 10, stop of 20, 20 x 20, 10 x 40, and 20 x 40), you will always find the peak value. For a test of this size, if these were random seeds, three values that gave the same results would be enough. For laiger tests, more seed values would be needed.

More on Test Continuity

Any sjstem can be tested using a computer, but automatic selection of the best results only makes sense when they are continuous. A moving average sjstem with stops is continuous in all directions. Adding a third dimension, an entry or exit delay (0, 1, or 2 dsjs) will produce continuous results in the three directions. But not all tests satisl this principle. It has readj been briefly mentioned that rule changes are the most likely cause of discontinuity Consider a test in which one variable is the speed of the trend and the other is a set of rules, L, S, and B, where L only takes long positions, S only takes short positions, and takes both long and short positions. Although there may be interesting pattems and relationships, averaging these results does not make sense.

If each parameter tested has values that result in a continuous profit/loss curve, there are mathematical techniques available to locate the point of maximum profit with the minimum st s. One method, optimization by steepest descent, uses both the profit and the rate of change of profitability to ddermine the size of die st s taken during the optimization process. This method can be suboptimal, however, if there is more than one profit peak.