Editorial Reviews. Review. " an essential read."The Technical Analyst August From the Inside Flap. It would be an understatement to say that the world.
Table of contents
- The Evaluation and Optimization of Trading Strategies
- The Evaluation and Optimization of Trading Strategies : Robert Pardo :
- Wiley Trading
- Customers who bought this item also bought
- Frequently bought together
The Strategy Development Platform. The Elements of Strategy Design. The Three Principle Components of a Strategy. An Overview of a Typical Trading Strategy. A Trade Equals an Entry and an Exit. The Management of Risk. The Management of Profit. The Importance of Accuracy. Price and Trade Slippage. Opening and Closing Range Slippage. Slippage Due to Size. The Significance of Slippage.
Major Events and Dates. The Size of the Test Window. Sample Size and Statistical Error. The Life Cycle of a Trading Strategy. Window Size and Model Life. Make a Vague Idea Precise. Verification of Calculations and Trades. Determining the Length of the Test Period. The Results of the Test. The Prioritized Step Search. Hill Climbing Search Algorithms. Multi-Point Hill Climbing Search. General Problems with Search Methods. A Review of a Variety of Evaluation Methods. The Evaluation of the Optimization. The Robust Trading Strategy. The Statistically Significant Optimization Profile.
The Distribution of the Optimization Profile. The Shape of the Optimization Profile. How Does the Strategy Respond to Optimization? Does the Strategy Deserve Further Development? Is the Trading Strategy Robust? Robustness and Walk-Forward Efficiency. The Cure for Overfitting. Assessing the Impact of Market Changes. The Best Parameter Set for Trading. The Theory of Relevant Data. The Varieties of Market Conditions. The Role of the Walk Forward. Setting Up a Walk-Forward.
An Example of a Walk-Forward Test. The Purpose of the Walk-Forward Analysis. An Example of a Walk-Forward Analysis. Is the Strategy Robust? What Is the Risk? Walk-Forward Analysis and the Portfolio. The Evaluation of Performance. The Trading Strategy as an Investment. The Dimension of Risk. Compare the Strategy to the Alternatives. Maximum Drawdown and Trading Risk.
Maximum Drawdown in Context. Maximum Drawdown and the Trader. Maximum Run up and the Trader.
The Evaluation and Optimization of Trading Strategies
If the trading strategy is found to be satisfactory at the end of this first stage of testing, it is time to move on to the second round of testing, which is optimization. Optimization presents the proper methods to optimize a trading system. Optimization proceeds through two levels. The first is an optimization of the trading strategy over a variety of different markets and time periods.
The main purpose of this stage is to determine to what degree the trading strategy is enhanced by optimization. If the strategy demonstrates better performance under optimization, then it is taken to the final round of optimization and testing: Walk-Forward Analysis presents this advanced method of strategy optimization, testing, and validation alongside the three major objectives achieved by Walk-Forward Analysis WFA. The optimization of the trading strategy under an exhaustive WFA measures the trading performance exclusively on the basis of out-of-sample trading, that is, on data other than those used to optimize the strategy.
The first, and far and away the most important, objective of the WFA is to determine whether the trading strategy remains effective on unseen or out-of-sample price history. This, of course, is one of the most reliable and major predictors of real-time trading success. If it walks forward well, as we call it, then it is highly likely that it will continue to perform profitably in real-time trading.
The second and next most important objective of the WFA analysis is to determine the optimal parameter values to be used with real-time trading. The third objective is to determine the sizes of the optimization window and the periodic rate at which the strategy is to be reoptimized. Experience has only continued to prove its merit in the trading arena as the most cost-effective way of producing robust trading strategies that behave in real-time trading in a manner consistent with their historical simulations. Given its efficiency and practicality, it continues to be a surprise to me that it has not attained widespread acceptance and application.
It must be judged on its merits as an investment competing for capital with the entire universe of investments. It also must be evaluated in comparison to other available trading strategies on the basis of a statistical analysis and review of its own simulation profile and performance structure. The Evaluation of Performance presents these two essential, typically underappreciated, and often misunderstood procedures. The simple truth is that with contemporary trading strategy development software and the modern computer, it has never been easier to perform an optimization of a trading strategy.
The proper ways, however, to test, optimize, and evaluate a trading strategy are not necessarily well known by all of those in the trading community who use these applications. It is precisely because it is so easy to perform an optimization but so difficult to evaluate it correctly and then successfully trade it in real time, however, that the reputation of optimization has been unfairly tainted by those ignorant of its correct procedure and evaluation.
In fact, it is because of this widespread misuse of optimization that some still falsely equate the term optimization with the term overfitting.
The Evaluation and Optimization of Trading Strategies : Robert Pardo :
As we see in Chapter 13, overfitting or curve-fitting is really optimization done incorrectly, carelessly, or gone wrong in some other way. The Many Faces of Overfitting puts forth the proposition that the overfitting of a trading strategy to historical data occurs when testing and optimization are done incorrectly. The proper evaluation of an optimization can be a very difficult matter.
I personally believe that the most effective way to avoid overfitting during the optimization process is to perform optimization through a Walk-Forward Analysis. The effects of the overfitting or curve-fitting of a trading strategy to its historical data are devastating, and an overoptimized trading strategy often leads to significant and immediate real-time trading losses.
To help the strategist avoid overfitting, I dedicate an entire chapter to identifying the symptoms that result from the accidental abuse of proper testing and optimization methods. This chapter also includes an extensive discussion of a variety of methods designed to detect and avoid curve-fitting, including the most effective way to do this, which is to include out-of-sample testing in your optimization process. The goal of any trading strategy is to enjoy long-lasting, real-time trading profit. Once the full cycle of trading strategy development has been successfully completed—namely, strategy formulation, testing, optimization, walk-forward analysis, and evaluation—then, and only then, can real-time trading safely begin.
Trading the Strategy presents the guidelines one must follow to assess real-time trading performance in the context of the knowledge of profit and risk arrived at by computer testing and formulated in the statistical strategy profile. The improper evaluation of real-time performance will cause problems for the trading strategist. It is essential to know, within reason, that the carefully and painstakingly developed trading strategy is performing in real-time trading within the bounds of the trading strategy profile.
Without this essential knowledge, the strategist is like the captain of a ship at sea without any sort of navigational apparatus. Of course, it goes without saying that the goal or aim of trading is to cause our trading account to grow or to produce a profit. T Acting methodically according to a fixed plan that is designed to achieve a profitable return by going long or short in markets on organized financial exchanges. Let us recall from Chapter 1 that I take the terms systematic trading strategy and its short form, systematic trading, to mean, as in synonymous 17 c02 JWPRPardo December 14, 18 Of course, the overall aim of any trading strategy is the creation of wealth through trading excellence.
Trading excellence means the creation of the greatest rate of return possible with the least risk. Furthermore, trading excellence also means the reliable production of excellent returns with the greatest possible consistency from year to year and for the full duration of the trading life of the strategy. As stated earlier, the only purpose of trading is to produce profits. The main reasons that a properly tested and validated systematic trading strategy helps in the pursuit of trading profit are its: But, before we do that, let us explore the major advantages or benefits of the systematic trading strategy.
This chapter presents the philosophical and practical reasons why someone would choose to trade with a systematic strategy. It is the why of systematic trading. That being said, trading is a human activity, and we humans have a nearly inexhaustible number of reasons to remap and distort our intended goals, but that is a topic for a different book. This very tendency toward fallibility, however, is the other highly significant reason that having a systematic trading strategy helps produce profit.
The systematic strategy—unless overridden by well-intentioned but often misguided human judgment—does so without the emotion, fallibility, and error-prone guidance of the all-too-human trader. The properly designed and verified systematic trading strategy pursues trading profit with the relentless consistency and objectivity of computer logic.
The discretionary trader decides what to do each time he makes a trade. It is all put upon the shoulders, as it should be, of the trader. It is up to the skill, knowledge, experience, control, emotional balance, and discipline of the trader. Let it be known, even though I am known as a leading advocate of systematic trading, that I hold the successful discretionary trader in the highest regard. Discretionary trading demands the mastery of a number of demanding skills.
Aside from all of the technical skills that the discretionary trader must master, first and foremost, the successful ones of long-standing tenure are masters of themselves. Remember that the inability to follow a proven strategy is high on the list of reasons for failure of the systematic trader. How much more difficult must it be for the discretionary trader who needs to be on and in control of himself day in and day out?
Let us extend this idea a bit further. Consider how difficult it is for the systematic trader when undergoing a drawdown—even when it is in keeping with the risk profile of the strategy—to stay with the strategy and make the next trade, how much more so it must be for the discretionary trader to pull the trigger when faced with loss after loss. Certainly such a condition will have a very corrosive influence on his self-confidence. As we explore the advantages of the systematic strategy in more detail later in this section, the differences that emerge will be highlighted.
Let us consider the plus side of the discretionary trader. It is quite simple. The biggest plus is that, to date, I do not believe that a systematic strategy has yet been created that equals, let alone exceeds, the performance of the greatest discretionary traders. Proof of this concept is available by the mere consideration of a short list of some of the household names of the greatest discretionary traders.
This short list of the greatest would include the likes of legendary billionaires such as Warren Buffett I know he is not a trader per se, but he is the second richest man in the world and he got there solely through investing , George Soros, Paul Tudor Jones, Bruce Kovner, and T. A greater indictment—and there is a causal relationship here to a certain extent—is that commercially available trading strategy development software has woefully lagged behind on all fronts. The most significant difference between the performance of the highly skilled discretionary trader and that of the systematic trading strategy is merely one of degree and not one of kind.
The discretionary trader has a vast knowledge base of different trading methods and strategies. This knowledge base also holds a store of knowledge about, for example, the strengths and weaknesses of these different strategies as well as their interactions with one another and under different market conditions.
Such traders also have the benefit of finely honed reflexes and observational skills that can, sometimes nearly instantaneously, detect a complex pattern in one flash of insight that tells him that the market just made a top. If he is heavily long, he then gets out of all positions as quickly as possible. Of course, this is highly simplified, but the significant point is that the successful and experienced discretionary trader brings a vast knowledge of different methods of analysis, trading strategies, market knowledge, and pattern recognition to what he does.
All this knowledge is then sifted, filtered, and parsed by the human brain through a process of synthesis and experience to arrive at a proper and timely buy-or-sell decision. In contrast, consider a relatively simple but widely used systematic trading strategy made famous by Richard Dennis and the group of trading students, the famous Turtles, whom he trained in its use. Let us consider a highly simplified version of this strategy.
The Turtle Trading Strategy TTS is a range breakout method originally derived from a strategy developed by Richard Donchian, an early pioneer of systematic trading. TTS goes long when an x-day high has been penetrated and goes short when an x-day low has been broken. It exits positions on opposite signals from a shorter y-day high or low.
Exit long and short positions on opposite signals at y-day highs or lows. In its original form, it factored in no other knowledge or information. Anyone with any imagination can certainly think of a long list of other indications that might be added to this strategy for its enhancement. The point to be noted here, however, is not the relative simplicity of the strategy, although that too is significant, but rather the absence of any other information factored into this decision to go long or short. There is no reason to believe that a systematic trading strategy cannot be made that rivals the knowledge base of the discretionary trader.
Just because it has not been done yet, at least to my knowledge, does not mean that it cannot or will not be done. When computer scientists first began the development of computer programs to play chess, it was said that a computer would never beat a chess master. Evidence is mounting that the front-runners in systematic trading are finding ways to develop, test, and trade increasingly sophisticated and highly profitable systematic trading strategies. I created this concept for use as an important measure of performance for our implementation of Walk-Forward Analysis.
I presented this in the first edition of this book on page under a discussion of Model Efficiency. An expanded version of this discussion can be found in Chapter 12 of this book: The Evaluation of Performance. Consider, however, the potential Perfect Profit profit obtainable for five markets in five years on daily bars. According to the Barclay Trading Group, as of June , only 16 CTAs out of a universe of produced a five-year annualized compound rate of return of 25 percent or better.
That is a return on our account of percent! Now let us consider that there are more than 11, listed stocks, 7, listed mutual funds, and over different futures markets, and the potential becomes absolutely staggering. When we factor in the profit potential that is available on crude oil futures on 5-, , and minute bars and we realize that all markets can be traded in a wide variety of time degrees, and whereas potential profit is not infinite, it is vast in the extreme. It certainly raises the bar on what the aggressive trader might target for his next trading portfolio of strategies.
When we consider the extensibility to all markets and time frames that systematic trading makes possible, the enterprising and imaginative strategist might begin to see how it might be possible to begin striving for returns that would have been unimaginable 10 years ago. Now that we have had a look at what the return potential of future systematic trading platforms may aspire to, let us take a look at the concrete benefits of the systematic approach to trading.
Indeed, perhaps the first and foremost advantage of a thoroughly tested systematic trading strategy is the determination that it, in fact, has a profit potential. Another way of looking at this is that a successful and fully tested systematic trading strategy is in itself a proof of the trading concept. Without a reasonable estimate of the potential risk-adjusted reward of a trading strategy, it is impossible to know if it is worth trading.
Without a reasonable estimate of potential risk, it is impossible to know the true cost of trading with the strategy. How is it that we determine that a trading strategy has a positive profit expectancy? This term will be formally defined and detailed in Chapter 6: Why is it that the strategist goes through all of the work to painstakingly construct and evaluate a historical trading simulation just to verify the validity of a trading system?
The answer to that question is quite simple and most basic. We want to see whether it works, that is, that it produces a trading profit on past historical market data. We also want to develop an opinion of the likelihood that the trading strategy will produce real-time profits in a proportion similar to that of its historical profile. That, however, is an exploration for Chapters 11 and We are also very interested to see what the profit and risk of the trading strategy is over both ever-changing market conditions and different markets.
If we find that the trading strategy produces profit over a range of conditions necessary and a variety of markets desirable, but not necessary , we have further validation of the trading concept. We also know that we have a more valuable trading strategy. An objectively formulated set of trading rules can be translated into a format that can be understood by a computer. The programmed trading system can then be tested over years of price history and many different types of markets: When the trading strategy has undergone a full and exhaustive round of development and testing and the results are positive, we know that we have a profitable trading strategy.
We can capitalize a trading account and begin trading with a higher degree of confidence in a positive and lasting outcome. In contrast, let us consider what is likely to follow if we begin our trading adventure with a bunch of vague, inconsistent, and unverified ideas. First, such a configuration cannot be computer tested. We have only our faith in the merits of the trading ideas.
We have no solid knowledge of their past effectiveness or of their risk and rate of return. Such vague and unverified ideas, of course, can be traded. All that is needed is some trading capital, a clearinghouse, and an account. The results of such an approach, however, are well-documented and predictable. As I have previously suggested, even the discretionary trader trades with a plan and systematically determines entries, exits, trade size, and so forth. Trading without a well-defined plan has the same likely outcome as doing anything without a plan—failure. Would you start a business without a plan?
Build a house without architectural and engineering drawings? Begin a journey of 5, miles without a road map? Why then would anyone begin trading without a trading plan? It reminds me a of a W. When a trading strategy has successfully undergone the full testing cycle from start to finish, it has been verified to have a positive expectancy. It has also been verified that the trading strategy has a reasonably high likelihood of producing real-time trading returns relatively consistent with its historical simulation.
Armed with this knowledge, the trader has a sound and rational basis for confidence in the trading strategy sufficient to trade with it and to follow it faithfully. An accurate measure of profit and the risk required to obtain it are needed for two main reasons. The first is to determine whether the risk-adjusted reward is equal to, inferior to, or superior to other competing trading and investment vehicles.
The second is to determine the optimal account capitalization required to obtain the maximum rate of sustainable return. Another tremendous advantage of the quantification or statistical evaluation of a trading strategy is that it makes it possible to accurately compare different trading strategies to one another. Because of the varying profit and risk profiles of different trading systems and the profit potential of different markets, the only thing that can be meaningfully compared from one system to another is the rate of its risk-adjusted rate of return.
It also provides a large number of other very useful statistics, such as the number of trades, the value of the average trade, statistics about winning and losing runs, and trading performance on a year-by-year basis. These statistics collectively compose the performance profile.
Perhaps one of the most important benefits of the historical trading profile is the proper evaluation of returns. Profit cannot be evaluated without a measurement of the risk that earning it entails. System is XTct 38, 38, 3 Performance Summary: This is not a terribly profitable system. Yet, neither is it a very risky one. This makes the system very appealing on a risk-adjusted return basis. Strategy One produces a risk-adjusted return of Based solely on its profit, this looks like a very profitable system.
However, it just turns out that it is an equally risky one. Its reward-to-risk ratio of 1 to 1 is very unappealing. Strategy Two produces a risk-adjusted return of Which strategy is better? If we are to judge correctly by the respective risk-adjusted return, however, or what it costs to earn a dollar of profit, Strategy One produces a return that is more than 6 percent better.
Profit and risk are inextricably interrelated. The trader or investor cannot have one without the other. It is analogous to an X-ray or an MRI scan of a person. More important, it provides both a road map and a benchmark providing what should—and, more important, what should not—be expected from its real-time trading performance.
- 10 Great Cocktails Ebook;
- On These Silken Sheets?
- !
- Top Authors!
This is discussed in detail in Chapter Suffice it to say that this performance profile is essential to the proper management of trading a systematic strategy. More important, without a statistically reliable measure of risk, it is impossible to manage future trading risk. Of course, this makes rational trading impossible. Risk is the primary dictator of the cost of trading a strategy.
Furthermore, without the proper measurement of risk, portfolio management is also impossible. This profile provides further insight into what type of performance to anticipate during differing market conditions, such as an uptrend or a downtrend, or during conditions of high and low market volume and volatility. The quantification of the trading strategy also needs to be extended over different markets.
The reward-and risk-measurements of individual markets are essential to the design and creation of a basket of markets or a trading portfolio. Without these individual market-risk-adjusted returns, it is impossible to determine appropriate market allocations and to balance a portfolio to achieve maximum risk-adjusted returns. The Trading Advantage of Quantification. The quantification of the risk and reward of the trading strategy provides the mathematical basis for the correct capitalization for real-time trading of the strategy.
The statistical performance profile that is also a result of this quantification provides a set of milestones by which real-time trading performance can be evaluated. Not dependent on the mind for existence. Furthermore, the objective, systematic trading strategy is external to the mind of the trader.
Wiley Trading
In other words, it has no reliance on the mind, and its highly varied states, of the trader for correct execution. Once specified and verified, the objective systematic trading strategy has a life of its own, like any other human creation, independent from its creator. There is nothing wrong with emotions—after all, constructive emotions are the drivers and wellsprings of life. Undisciplined and poorly understood emotions as trading inputs, however, are typically quite unreliable. All the more so because the typical emotions that play heavily, usually wreaking havoc on trading, are fear and greed.
Lack of confidence, or self-doubt, is another emotion highly antithetical to trading profit. As we all know, the emotion of fear triggers the fight-or-flight response in our biological system. If we choose to fight, we dig in. What does a trader do when he digs in? He stays with a trade, for better or for worse. If we choose flight, we run away from the threat. What is the flight response in a trader? We exit the trade typically when it is going against us , regardless of whether it is the right thing to do. What is the typical effect of greed on the trader? Typically, greed motivates a premature trading exit: The other typical, although opposite, response is to hold on to the trade too long.
The thinking runs something along these lines: There is an old trading proverb that sums this up rather well: The typical effect is chronic second-guessing and self-questioning to the point that a trade might not be taken or the position size reduced in a manner inconsistent with his trading rules. What is the typical result of such omissions? Often, the trade not taken is a winner and the downsized trade is a big winner. The destabilizing effect of undisciplined emotion is the introduction of tremendous inconsistency.
Furthermore, fear, greed, and self-doubt can have a tremendously negative impact on trading. These effects can be compounded by the likelihood that such emotions will be affected by whether the trader is healthy or sick, well-rested or exhausted, hungry or satisfied, happy or sad, tense or calm, and so on. Conversely, objectivity, with its independence from the mind, emotions, and opinions of the trader is the ally of consistency. The Trading Advantage of Objectivity. The objective systematic trading strategy is external to and independent from the mind, mood, and emotions of the trader.
Once formulated and thoroughly researched, the systematic trading strategy has a life and independence of its own. Furthermore, the trading strategy is independent from the whimsy of and buffeting by the often-turbulent emotions of the trader. This objectivity and freedom from the inconsistent whimsy of the trader promotes consistency in trading.
The systematic trading strategy, if followed without exception, provides its own form of trading discipline. Consistency, in fact, is arguably one of the major influences contributing to profitable trading. Consistency in trading means the application of the same trading rules—entries, exits, trading size, risk management, and so forth—for each and every trade produced by the trading strategy.
The execution of a trading strategy automatically in real-time trading will produce profits and losses that are relatively consistent with the results of its profitable, verified, and robust historical simulation. The alteration of the rules in the execution of a systematic trading strategy will not.
The optimization of the trading strategy produces a robust set of model parameters, which hold reward and risk in a delicate balance. It is certainly true that the more robust and excellent the trading strategy, the less delicate this balance is. But even the most robust trading strategy can and will be eventually ruined by the acceptance or rejection of trading signals that can and will result from human interference.
Only after the risk of a trading strategy has been properly identified and measured can it then be successfully monitored and managed in real-time trading. It is impossible to manage risk successfully when it is undefined or constantly changing. A typical form of trader interference is to not take a trading signal. In brief, this usually has to do with fear or a lack of self-confidence. The net effect, however, of not taking every trade produced by a strategy has the effect of transforming the precisely measured risk, which is balanced with reward in that delicate equation arrived at in the development process, into an unknown and moving target.
To illustrate the effect of risk as a moving target in a more concrete manner, let us consider hypothetical System X. Now let us consider System Y. Y is a variant—perhaps an aberration—of X. This would be great.
It could also be worse. As this simple example illustrates, ignorance of the risk of a trading strategy makes it impossible to properly assess the return of the strategy. Even worse, such ignorance of risk makes it impossible to properly capitalize the trading strategy. Obviously, no one ever really knows whether the next trade is going to be a winner or a loser. The only good reason, however, why a trader would choose to take or not to take the next signal would be foreknowledge of its outcome.
That, however, is impossible. The real effect of such picking and choosing of signals is to destabilize the delicate balance between profit and loss uncovered during the strategy development process. Conversely, the ones taken are typically losers. Let us get back to our metaphor of risk as a moving target.
Extending the logic developed in the previous paragraph, missing winners and picking the trades that lose will wind up affecting the balance of profit and loss by decreasing profit and increasing loss. Another typical form of trader interference with the systematic trading strategy can extend to trade sizing, and this can, and quite rapidly, lead to potentially devastating results.
Consider the following example. The trading strategy is on a hot streak and it has produced seven wins in a row and decent size ones, too, trading a unit size of two contracts per trade. Feeling flush from the extended winning streak and the big equity increase, the trader feels like the strategy can do no wrong and increases the unit size to ten contracts per trade. Well, anyone who has traded for any length of time knows that all winning streaks come to an end. So does that of our hot-handed trading strategy and its does so with three typically sized losers in a row at the larger trade size of ten contracts.
Of course, a trade size five times larger than that which produced the winning streak will lose money at a rate five times greater. As a result, the trader winds up giving back all of the profits from the last winning run and much more as well. Without consistency of entry, exit, risk, and trade size on each and every trade, it is impossible to estimate the probability of success. It is essential to understand that without a thorough knowledge of strategy-specific risk, it is impossible to intelligently trade with any strategy, whether automatic or discretionary.
Consistency also means knowing in advance how to act in any circumstance, based on preestablished and verified rules.
- By-catch Reduction in the Worlds Fisheries: 7 (Reviews: Methods and Technologies in Fish Biology and!
- I am a Fisherman on Lough Carra.
- Kennedy: The Classic Biography (Harper Perennial Political Classics);
- Buy Both and Save 25%!;
Contrast this to the inconsistency and unpredictability of trading responses based on emotions such as fear and greed. Consider the tremendous advantage this confers during those infrequent but oftentimes very dramatic, fast, and either hugely rewarding or painfully costly price moves that occur from time to time driven by sudden, large, and unexpected political or economic events. The Trading Advantage of Consistency. A sound systematic trading strategy is that it consistently—with the uncluttered logic of mathematics and the relentlessness of computers—applies the same entry and exit rules without exception or deviation.
As a result of this, and assuming a relative consistency in market activity from period to period, which is not always the case, the size and frequency of profits and losses will remain reasonably in line with that of its performance profile. If we replace market with markets and time frames, we have yet another important application of this concept. If we further add systematic trading strategies, then one can really begin to see the considerable advantage conferred by the extensibility of trading strategies through total automation. One of the most valuable but often overlooked advantages of the systematic trading strategy is that, given sufficient trading capital and computing capacity, it can be applied to as many different markets and time frames in which it has proven itself to be effective.
Taking this logic one very important step further, and given the same provisos, a group of different systematic trading strategies can be traded over a portfolio of different markets and different time frames. Given the rise of global markets, trading is now possible almost 24 hours a day and 7 days a week. This is, of course, very expensive. But consider how much more cost-effective, quick, and reliable a computerized installation of such a vehicle would be.
Such a computerized trading desk would only be limited by its hardware and communication capabilities. It should also be noted that such highly computerized trading desks already exist. Now consider the recent and rapid rise and availability of electronic trading in the vast majority of markets and which many experts believe will soon include all markets. Combine this with the rapid rise of automatic order entry and we are looking at the perfect climate for the systematic trading of a basket of systematic trading strategies, over a basket of markets and time frames.
And, once again, rest assured that such highly systematic trading is also taking place. By no means, however, is it allpervasive—at least as of yet. The Trading Advantage of Extensibility. A profitable and sound systematic trading strategy run by computer makes it possible to trade virtually as many markets for which the trader has capital. The availability and effectiveness of automatic order entry makes this an increasingly realistic proposition. It is now technically possible to trade a highly complex systematic trading strategy through a computer and with no human involvement other than that of its creation, testing, and implementation.
How is a trading simulation produced?
Two things are needed: The first step, then, is to create a precise formulation of all of the rules of the trading strategy in a computer-testable language. Next, this strategy formulation is then processed by a computer application—a trading simulator—that has the ability to trade, or to apply, this strategy on historical data. The trading simulation software then collects all of the trades—buys, sells, and individual profit and loss—produced by the strategy during this historical period and a number of different statistical performance reports are created from them.
The level of detail and the different types of information contained in a historical trading performance varies from application to application. There are a number of common statistics, however, that are included in all such reports, including, for example, net profit, maximum drawdown, number of trades, percentage of winners, and average trade. Producing a historical simulation with the proper software is a straightforward process. Once the trade simulation software has been mastered and a historical database of prices acquired, producing trading simulations is a relatively simple process.
Deducing that the trading strategy has a positive expectancy is also quite simple: Either the historical simulation is profitable or it is not. However, and this is a big however, determining that the historical simulation has produced a result that is reproducible in real-time trading is quite a different matter.
Now that we have a basic idea, however, of what a trading simulation is and what information it provides, let us go on to explore the various benefits that accrue from the creation and examination of a trading strategy.
Customers who bought this item also bought
If the results of the simulation are profitable, at least we now know that we have something that traded profitably in the past and something with which we can work to determine whether it will produce such returns in future trading. An accurate historical simulation of the trading strategy is the only way to determine whether the strategy has a positive expectancy. The alternative, of course, is to just start trading with the untested strategy in real time.
I would not recommend this latter course of action, though. The historical simulation and its evaluation can and must answer two very important questions. First, the strategist needs to determine how effective the trading strategy has been historically. This needs to be evaluated as an investment comparing it to a variety of competing alternatives.
Frequently bought together
This is explored in Chapter This is a far more difficult determination to make. The Many Faces of Overfitting. Furthermore, if one uses Walk-Forward Analysis to optimize and validate the strategy, arriving at this determination with a high degree of confidence is far more straightforward and mechanical. It is one of the central theses of this book that Walk-Forward Analysis is the most effective remedy for, and method of, avoiding overfitting.
The historical simulation of course, is the mode of operation of Walk-Forward Analysis. In fact, as we see in Chapter 10, WFA adds another level of simulation to the trading strategy process. Of course, it all begins with and could not proceed without the historical simulation. It is a complicated and sometimes difficult process to simulate, optimize, and evaluate a trading strategy from start to finish correctly.
We have seen how easy it is to create a historical simulation of a trading strategy. The accurate evaluation of the validity of the historical simulation however, is not quite so simple. Also, to establish with a high degree of confidence that a trading strategy will work in real-time trading is a complex and more difficult process.
As we will see, one of the greatest benefits of Walk-Forward Analysis is the bright light that it sheds on this question. The first is to use this information to determine the minimum proper capitalization of the c02 JWPRPardo 36 December 14, The pursuit of consistent, high, and sometimes outsized trading profits is what drives the trader to trade.
It is the size of the risk, however, that tells the trader how much it is going to cost to achieve this profit. Reward cannot be properly evaluated in the absence of its attendant risk. The only meaningful and practical measure of performance is in the form of the risk-adjusted return. This central statistical measure of performance is defined and detailed in Chapter What concerns us here is that this measure is central to the proper capitalization of a trading account.
Proper capitalization means that an account is funded with sufficient capital to absorb the maximum risk, or drawdown, that the trading strategy may endure. More important, not only can it absorb this drawdown, the account must have sufficient capital remaining after drawdown to continue to trade with the strategy. With a properly developed systematic trading strategy, real-time trading performance should conform in general to that of the statistical profile of performance. This is unlikely to be an exact correspondence, of course.
This is explored in detail in Chapter If real-time performance is dramatically different from that of its performance profile, however, whether on the side of profit or loss, then we must satisfy ourselves as to the reason for this discrepancy. If a valid reason cannot be found, then it is even more important to be highly vigilant until trading begins to realign itself to the performance profile. Be aware that strategies that have perhaps gone awry do not always return to normalcy. We are all too quick, of course, to consider abandoning a strategy if losses begin to exceed our expectations.
This is not always the correct thing to do, however. Conversely, we are laughing all the way to the bank when our strategy is more profitable than its performance profile. And, of course, there are valid explanations for windfall-size profits. The sword of volatility cuts both ways. In summary, the benefits of the historical simulation are a: One of the main reasons that we optimize a trading strategy then, is to obtain its peak trading performance.
As we see in the next section, Walk-Forward Analysis takes this one step further. Most trading strategies have rules that accept various different values. For example, a two moving average crossover system will have a value for the length of each of the averages. Whether the analyst chooses to optimize these values, any strategy that can accept different values for its rules is optimizable, if so desired.
The first function of optimization then, is to determine the appropriate values for the most robust implementation of the trading strategy. Just as the creation of a historical simulation with trading simulation software is trivial, so is optimization. Doing it correctly, however, and determining the most robust parameter set is nontrivial. The second, and more important and more difficult, function of optimization is to arrive at an estimation of the robustness of the trading strategy.
Let us just note here that robustness, or the lack thereof, of a trading strategy can show itself in this stage of strategy development. Anyone who has been around trading for any length of time knows that market conditions are in a relatively constant state of flux. Trends end and change. Volatility rises and falls, as does trading volume.
Political and economic events add small, medium, and sometimes gigantic shocks to the markets. Change is one of the defining principles of a market. Hence, it must be a fundamental axiom of trading life that markets change, and as traders, we must be flexible and adapt to this change. Another of the main functions of optimization, and more so of WalkForward Analysis, is to arrive at and maintain peak trading performance in a systematic trading strategy in the face of this continual change.
It is also a function of optimization and Walk-Forward Analysis to adapt a trading strategy to different types of markets. All markets have their own unique personalities. A trading strategy may perform well in one market with one set of values and poorly in another with those same values. I do not believe that one set of parameters should necessarily be sufficient for every market in which the strategy is traded.
In my experience, such a situation is rare. In fact, I tend to prefer different values for a trading model for different markets, in that it offers an additional dimension of portfolio diversification. Optimization will identify the best set of parameter values for each market. Finally, different traders have different trading capital, time available, computing resources, profit expectations, tolerance for risk, and temperaments.
Another application of optimization then, is to adapt the trading strategy to the individual needs of the trader. The degree to which this is possible will vary with the style of the trading strategy as well as with the market to be traded. In summary then, the benefits of optimization are the: For our purposes here, it is sufficient to note that a Walk-Forward Analysis is a systematic and formalized manner of performing what has been referred to as a rolling optimization or a periodic reoptimization.
One of the primary benefits of the Walk-Forward Analysis is to determine the robustness of the trading strategy. Another important advantage of Walk-Forward Analysis is to produce peak trading performance as markets, trends, and volatility change. Since the Walk-Forward Analysis provides a method of periodic reoptimization with current price action, this often means that it can produce trading performance superior to that of traditional optimization. Since this periodic reoptimization is done with a strategy-appropriate amount of current price data, this also provides an efficient way to continuously adapt a trading model to ongoing changes in market conditions.
In summary, the main benefits of Walk-Forward Analysis are the: These benefits flow directly from the thorough, intimate, and precise understanding of the trading strategy that is a product of this approach to trading. These five important benefits are: A high degree of confidence that your strategy will perform in real- time trading as it has in historical simulation 3. The confidence to stick with the trading strategy during good times and bad 5. A comprehensive knowledge of the trading strategy and of its real-time trading performance, which makes it easier to successfully improve and further refine the trading strategy over time c02 JWPRPardo December 14, 40 We have seen how this approach leads to: The most solid confidence is one based upon knowledge and understanding.
Gann had a saying of which I have always been particularly fond. The most important knowledge which this process produces is of: He does not need to hope that his strategy will work because he knows that his thoroughly evaluated strategy offers the best possible prospect for trading success. The bottom line is that the solid knowledge that results from a thoroughly tested and well-understood trading strategy produces a tremendous and well-founded confidence by the trader in his trading strategy.
The main benefit of this well-founded confidence is that it increases the likelihood of the trader being able to follow his strategy absolutely and to the letter of the law. Conversely, deviation from such a well-tested trading strategy will most often lead to trading loss. Most important, such knowledge-based confidence makes it much easier for a trader so equipped to stay with the trading strategy during its inevitable lean periods of losing streaks, drawdowns, and low returns.
The trader so equipped trades with knowledge, not with false hope. For example, consider a trend-following system. Assume that the driver of the strategy is the breakout of a volatility channel around a moving average. As the creator and trader, you have observed a tendency for the signals to have an appreciable lag behind market action.
Knowing this, you can search for other patterns or indicators that may be able to reduce this lag. Consider a countertrend strategy. Assume that the driver is some measure of an extreme reading in an overbought or oversold indicator such as a stochastic oscillator. As the creator and trader of this strategy, you have observed a tendency in real-time trading for the best signals to occur at intermediate-term cycle tops and bottoms. With this information, the developer can research cycle detection software as a potential way to filter the trades taken by the strategy.
In short, the development and application of a trading strategy follows eight steps: Specification in computer-testable form 3. Evaluation of performance and robustness 6. Trading of the strategy 7. Monitoring of trading performance 8. Refinement and evolution Each of these topics will be developed in its own chapter. This chapter comprises two parts. The first is an introduction to the philosophical orientation taken in this book. The second is an overview of the development process so as to provide an overall context within which to place the very detailed material in the subsequent chapters dedicated to these topics.
The first approach applies reason in the original design and conceptualization of the trading strategy. This is followed by the systematic and empirical verification of each component of the trading strategy. Every element of the strategy must make sense before the testing process even begins. I refer to this as the scientific approach to strategy development and it is the approach that I primarily follow in this book and in my trading.
The second approach might best be called the empirical approach. To a large extent, the logic and reason of the strategy developer is eclipsed, and to a varying extent, replaced by computer intelligence. The empirical approach uses various forms of computer software technology to search a space comprising a vast library of indicators, patterns, and price action to assemble a profitable set of trading rules that become the trading strategy if accepted and put to use.
In other words, the computer picks and optimizes a trading strategy developed in this manner. Though the fields of artificial and machine intelligence have come a long way, I do not feel that there is evidence that they are yet up to a task of this magnitude.
The main drawback to this approach is that it is relatively easy for this to devolve into a morass of sophisticated overfitting. Also, the actual logic of the trading strategies that emerge from this process is often not visible to the trader. The trader is, typically, and with good reason, unwilling to invest millions of dollars traded on a strategy that is essentially unknown to the trader. The Scientific Approach It might well be said that applying the scientific approach to trading strategy development follows the scientific method.
Namely, one formulates a hypothesis and then tests this theory to verify whether it is true—profitable and robust—or false—unprofitable and untradable. Whether the original theory has been found to be true or false, a dialectical process of further development and refinement can begin. In this process, new theories the refinements to the original are formulated and then accepted or rejected as true or false.
The scientific approach to trading strategy development is the methodology that is detailed in this book. The trading models that emerge from the scientific approach offer an invaluable benefit because the trader of such a system completely c03 JWPRPardo December 14, A theory of market action is proposed that states that a bigger-than-usual momentum surge predicts the beginning of a new trend in the direction of the surge. The strategist formulates a means of trading and verifying the theory. The strategy would stipulate: Exit on opposite signals. MSS is thereby found to be modestly profitable over five different markets each over five two-year historical periods.
Optimization and Walk-Forward testing of MSS uncover the full scope of its profitability, it is found to be sound, and it is traded in real time. Because of thorough testing, real-time performance follows test performance, so the trading continues. Furthermore, the trader is able to continually refine the system. In my experience, the vast majority of traders employ the scientific approach. It is also the approach that I have followed with much success and satisfaction over the years.
The Path of Empirical Development The onset of this second approach to strategy creation largely results from the development and refinement of a large array of computerized tools such as neural nets, data mining, genetic programming, and the other forms of computer artificial intelligence technology that are often referred to as machine intelligence. It is beyond the purview of this book to explore the pros and cons of this method in the detail that it demands. In fact, a thoughtful analysis of that topic could easily result in a lengthy tome of its own. Therefore, let us consider a much simplified and somewhat crude example of this approach that is inadvertently employed by many uninformed traders attempting to find a tradable strategy.
As an example, imagine that a trader is dabbling with a strategy development program that allows for easy profit and loss results along with a whole array of statistical measures on any trading strategy entered into c03 JWPRPardo 46 December 14, Although he tinkers with this formula found in a book, its workings, benefits, and pitfalls are not fully understood. One or two of the 30 variations of this formula show marginal profit on one period of history in one market and the trader thinks this looks promising.