System Analysis

We begin our evaluation with the bottom line. How profitable was the system during the trading period? Some traders believe that the bigger the profit, the better the system. Although important, net profit is only one measure of a systems worth. How a system makes its profit can, in certain instances, be far more important than its actual bottom line.

As an example, look at the results of the two trading systems in Table 9.2. System A made $350K while System made $375K during the trading period. Although System A made $25K less profit, most traders would agree that it is still the better system. This is based on System As consistent trading compared with System Bs volatile trading. The pain and suffering associated with System is simply not apparent from its final net profit.

Valuing a systems performance based solely on its net profit can be a mistake. With todays high-speed computers and advanced trading software, almost anyone can create a winning trading system. After a few optimization runs, almost any system will produce a positive net profit based on historical data. The hard part is designing a system that actually trades well in real time and that is not curve fit to maximize net profit.

The remainder of this chapter centers on evaluation tools that look beyond the bottom line. We will focus our attention on how a system generates its net profit based on multiple reward/risk measures. At the end of a systematic evaluation, a trader should be able to answer the following questions:

• Do you over- or underestimate your systems true performance?

• How stable are your winning and losing trades?

• What is your systems pessimistic reward/risk ratio?

• How many of your trades are statistical outliers?

• How does your system compare with a buy-and-hold strategvL.

• What is the average run-up and drawdown for the system?

Table 9.2 Annual net profit and loss of two systems

And most important. . .

• Do you really know your trading system well enough to trade it with complete confidence?

The next few sections will help traders answer these tough questions to fully quantify a systems true strength and weakness.

Profit Ratios

The following ratios use net profit as a building block for more detailed oriented evaluation tools. To fully appreciate the evaluation tools listed in Box 1, lets take a look at two trading systems. The systems in Table 9.3 made the same dollar amount but their ratios tell a slightly different story. System A had a higher Profit Factor, Adjusted Profit Factor, and Average Win/Average Loss than System B. With all things equal, System A is the more efficient trading system.

This analysis is based on reward/risk ratios. In this case, reward is measured by gross profit while risk is measured by gross loss. Each of the ratios listed in Box 1 uses these reward/risk measures as a basis for calculations. Theoretically, the higher the ratio, the more efficient the system.

Every trader wants to maximize profit; successful traders maximize profit in relation to risk. Profit ratios set a higher standard of evaluation by weeding out the inferior trading systems.

Return Measures

The next step in the evaluation process is reviewing trading performance based on return. Do the systems profits justify trading the system? These results allow for

Box 1

PROFIT RATIOS

Profit factor: Gross profit divided by gross loss. This calculation represents how much money was made for every dollar lost. Look for a system with a profit factor of 3 or more.

Adjusted profit factor: This tool artificially deflates winning trades and inflates losing trades. The net result is a more pessimistic profit factor. Look for a system with an adjusted profit factor of 2 or more.

Ratio avg. win/avg. loss: Average winning trade divided by average losing trade. Look for a system with a ratio of 2 or more.

easy side-by-side comparisons between systems. System returns can be measured in several ways (see Box 2 for more information).

Return on initial capital, also referred to as return on account (ROA), is the primary measure that best evaluates trading performance. It is calculated by dividing net profit by the systems initial starting capital. A system that makes $1,000 based on initial capital of $10,000 nets a 10 percent return. However, under the same conditions, if initial capital was $100,000, the same profit would only net a 1 percent return. To accurately compare the trading performance of systems running on the same historical price data, the same initial capital must be used. Surprisingly, traders often use different starting values in their comparison not realizing the inaccuracy of their evaluation.

A systems annual return also serves an important purpose in the evaluation process. It adjusts the return on initial capital figure and presents it as an annual figure. Judging performance on an annual basis creates an even playing field to compare trading systems.

Other return calculations relate the system to its own buy/hold return. If a system fails to outperform its own buy/hold return, then questions should be raised. For example, if a system trades 100 percent of the time and nets 15 percent in a year, is it good or bad? The answer actually depends on how the underlying security behaves in a simple buy-and-hold strategy. If the security buy/hold return nets 20 percent during

BOX 2

RETURN MEASURES

Return on initial capital: Net profit divided by the systems initial capital.

Annual return: Return on initial capital divided by the test period in years.

System return: Net profit divided by the initial purchase price expressed in percent.

Buy/Hold return: The most recent price divided by the initial purchase price.

Table 9.3 Profit ratios