Backtesting Commodity Trading Systems

Backtesting any commodity trading system or strategy is essential prior to trading it with real money. Our objective is to ensure the strategy is robust over the different market environments – bull and bear (both quiet and volatile) and trendless.

To accomplish this, we want to backtest the strategy over as long a time period as possible – 36 years or more. The reason for this is we want to make sure each of the different market environments are present in our test results.

By backtesting on too short of a period, we will avoid entire periods of certain market environments. This can result in surprising us greatly if we encounter a market shift while we are trading our strategy.

We require approximately 36 years of data because a commodity bull market lasts on average 18 years. So, to capture an entire bull to bear cycle requires approximately 36 years of data. We should also run our test on as many liquid markets as possible. Backtesting on illiquid markets corrupts our test results due to the large and frequent price gaps.

After backtesting our commodity trading system over a long period of time, we then want to break down our testing period into smaller intervals. This will lead to more specific results for each time frame.

Below is a partial test result for a commercially available futures trading system on 32 liquid markets. From the initial results, the system looks pretty good.

From 1980 – 2005 (26 years), the average trade was $599 and the winning percentage was 41.2%. The average initial trade risk was $1,669. The E or Expectation was 35.9%, which means the system made 35.9 cents for every dollar risked.

The average annual dollar return for this period was $76,672 (128 x $599). The total equity risked was $213,632 per year (128 x $1669). Funding the structure with the $213k would have, of course, produced the E or 35.9 % return per year.

These are solid results. Most traders, looking at only the above results, would trade this system with no qualms whatsoever. However, let’s now break out the testing period and see what happens.

As you can see, the results in the last time period were abysmal. The average trade was down 75% from the original test. The average trade risk increased by 21% or $358. This resulted in risking on average $320,266 per year over the 2000 – 2005 period. Quite a bit more!

Without breaking out the testing period, you would have never seen these results and worse… this is the period you would have started trading in – the latest period of the test.

So, what happened to this system? Did it break? Did the market environment change? Actually, this was an open source coded system. Not only did a massive amount of money start trading the system, but a separate contingent of money started fading the signals in the illiquid markets in the portfolio. Note, that our test does not even include those illiquid markets - or the results would have been a lot worse. The system went on to experience a drawdown larger than 60%. Ouch!

It is pretty obvious that if we do not backtest a commodity futures trading system on enough data and over a long enough time period the results will have very little, if any, significance in predicting future performance.

This example illustrates that when backtesting over long periods, the results can be skewed by superior periods of performance (1980-1990), watering down poor periods of performance (2000-2005). This is why you must backtest over several time periods within the original test.

Some developers like performing “out-of-sample” and “walk-forward” backtesting. This is basically when you test your futures trading strategy on only part of your data and then attempt to confirm your results with the remaining data. We feel the problem with this testing is the initial test is often not done on a long enough time period and that this kind of testing lends itself to over-optimization.

At Commodity Trading Solutions, we pride ourselves in backtesting our strategies rigorously. Our real-time performance attests to that.

CFTC REQUIRED RISK DISCLOSURE
Futures and options trading involve substantial risk. The valuation of futures and options may fluctuate, and as a result, clients may lose more than their original investment. In no event should the content of this website be construed as an express or an implied promise, guarantee or implication by or from Commodity Trading Solutions, LLC that you will profit or that losses can or will be limited in any manner whatsoever. Past performance is not necessarily indicative of future results. Information provided on this website is intended solely for informative purposes and is obtained from sources believed to be reliable. Information is in no way guaranteed. No guarantee of any kind is implied or possible where projections of future conditions are attempted.