Our Systems vs. Others
At any given moment there are hundreds of trading systems that are available to the public. While there are indeed a few good ones out
there, most trading systems will eventually fail. Unfortunately that is a fact. If you are like us and have tried over a dozen systems from
numerous developers, only to be disappointed with most, (needless to mention the financial losses), then by now you probably know what
to avoid. However, many among the unsuspecting public continue to fall victim to either unscrupulous developers, or systems that were
developed by developers with good intentions, but somehow in the development process something went wrong and from the day it was
born (released to the public), it already had the "genetic code" to blow up.
Regardless which system you go with, to help you in your search for a good system that puts the odds in your favor that over the long
term you will end up making money, here are a few thoughts to consider. Few systems will make it through the bullet items that we have
listed below. Those that do are systems that we would trust and trade ourselves, otherwise beware!
- Drawdown Analysis: all systems go through drawdowns (losing periods), yet many systems focus only on profit potential and minimize
attention to drawdowns. A detailed drawdown analysis is necessary for it is during the drawdown periods that a system is truly tested.
Furthermore, when comparing two systems side by side, drawdowns must be as equally important as profits (if not more). Rather than
comparing systems based solely on profits, they should be compared based on profit to maximum drawdown ratios. Furthermore this
should be done on identical portfolios consisting of the exact same markets, tested over the exact same time period and with the exact
same deductions for commission & slippage applied, i.e. all criteria under the exact same conditions. Only then can you get a true "apples
to apples" comparison. If System A makes twice as much money as System B, but also incurs twice as large a maximum drawdown, then in
reality both systems perform the same based on profits to drawdown ratios, hence in terms of percentage returns they will be the same,
however System A will require twice as much capital to start with due to the larger drawdowns. So don't forget drawdowns! Any good
system will include a detailed drawdown analysis.
- Consistency of Equity Curves: mere numbers are not enough! Remember the old saying that "a picture tells a thousand words". Equity
curve images are essential for they will show how consistent or "steady" the system's performance truly is. Absent an equity curve you
don't know if most of the system's profits were made on only a few trades on a few markets over a few years and the rest of the time
performance was flat or negative. Some unusual market conditions or situations may have presented themselves in certain markets over a
particular time period and that was when all the money was made. The problem is that such unusual situations might a long time to
reappear or never happen again! Hence evaluating systems absent an equity curve is asking for trouble. An equity curve reveals a lot
about the true nature and performance of the system. Furthermore, while no equity curve will be a perfect straight line, the smoother the
equity curve is the better, i.e. the smaller its peaks and valleys are the better. The smoother an equity curve is the more consistent the
system's performance is. Don't even consider systems without first inspecting the equity curve. If no equity curve image is provided walk
- Post-Release Performance: how long has the system been out? Post-Release Performance is the most important performance record
since this is data that was not available when the system was developed - it hadn't happened yet. Unfortunately the majority of systems
break down and go by the wayside a couple of years after release. When they are launched they come on with strong advertising claims
to back them up. Being the new sensation, the brokers jump all over them and push them on their clients. Everything starts off great the
first year, then Ok the second year. By the third year they are discontinued by the developer, or more likely some new version of them
comes out with a new name (where obviously they have been changed to keep the back-tested historical performance results looking
good). This is the typical pattern seen repeated again and again and again.
Sometimes developers will release subsequent versions with the excuse that market conditions have changed and thus the system
"needed updating". This is most likely tampering with the parameter values to re-optimize / re-curve fit the systems so that in hypothetical
historical performance continues to look good. Yet in other even more serious cases the developer will toss away the old system and
launch a new one every couple of years with a brand new name. Be very suspicious of this. If a system isn't broken then why fix it with a
new version every couple of years, or get rid of it?
- Historical Track Record: while it is obviously better to go with a system that has proven itself in post-release performance, obviously a
good new or newer system will not have this by no fault of its own. After all, a new system won't have much of a post-release performance.
In such a case, the only alternative remaining is to see how far back its back-testing historical results go. It is amusing to see systems that
only show the last couple of years or no more than 5 years! What about before then? How would the system have performed before then?
It leads one to be suspicious. We recommend at a minimum 20 years of data.
- Fully Disclosed / Totally Transparent Systems: this is the second most important item to consider after performance track records. The
vast majority of systems these days are sold in closed "black box" format, meaning that you the customer does not have access to the
logic, i.e. the trading rules, the source code. You are meant to blindly trust the system without knowing how it works, or perhaps in some
cases only "partially" knowing its algorithm logic. Developers of such systems claim that they cannot fully open up their system due to the
dangers of piracy and plagiarism that is so prevalent over the internet these days. They are protecting their intellectual property that they
worked so hard on to develop. Now this is a legitimate excuse - it is totally understandable and in some cases legitimate, however, it is also
a convenient excuse to produce a system that is massively optimized to the n'th degree and curve-fit. Then the system is packaged and
distributed as a black box and sold to the unsuspecting public.
We feel that the more a trader knows about the system the better. Yes there is the risk for us developers of falling victim to piracy,
plagiarism and copyright violations, hence it is a difficult choice for developers to make. Hardly anyone makes their intellectual property
totally open, no matter what it is, being a trading system or a gaming software. Nevertheless, if you are going to put the trader first, and
give him or her the benefit of the doubt, then clearly it serves the trader to fully understand what he or she is trading, vs. just blindly
following the trading signals that a black box system generates.
- Single vs. Multi Market Systems: this is another area where caution is warranted. Beware of systems that are designed to trade only
one specific market or just one specific sector. The risk is higher that once again such systems have been curve-fit, either intentionally or
inadvertently. While not profitable in every single market, a good system should show good results over a broad and diverse spectrum of
markets - always applying the exact same rules and parameter values across all markets! Otherwise, if the parameter values are tweaked
based on the market, what you really have is many "sub systems" within a system - one for each market, which once again leads to curve-
fitting. An example would be a system that let's say uses moving averages where the period used for the Swiss Franc is 29 days while for
Corn it is 17 days. Why the difference? Furthermore, if the system is a closed black box type systems (as most are), then there is simply
no way for you to know.
- Non-Optimized: this is a key point that we have been emphasizing all along. A system cannot be optimized for what worked best in the
past is not what will work best in the future. Markets constantly change. A good system built on solid concepts should not fall apart when
markets change, but should adapt. Yes there will be losing periods, known as drawdowns in the industry. All systems and traders alike
experience them, even the very best ones since nobody wins every day, but the good ones eventually recover and go on to make new
equity highs. If a system has been optimized to yield the best possible results based on historical past data, it is simply impossible that
those results will be consistently repeated in future data that has not yet happened.
When it comes to optimizing based on the market being traded, some will argue that each market has its own personality and
characteristics. After all, the 10 Treasury Year Note is very different than Cotton - right? Nothing could be more different. Yet how do you
explain those systems that do manage to produce profitable results across a broad & diverse spectrum of different markets, while always
applying the exact same rule formulas and parameter values for all markets?
- Simple systems with simple set of rules with few parameters: every now and then you come across a system that has all the criteria
mentioned thus far: good track record, fully disclosed, trades multiple markets without optimization, etc... then when you go to learn about
its trading rules or take a look at its source code to understand how it works it is like reading Martian! They are so complex and made up
of numerous convoluted formulas that only a PhD. in mathematics can decipher the logic. We believe in keeping things simple, for over
time simple systems have actually performed much more consistently than those complex ones. We believe in being able to explain a
system's logic in plain English to someone with nothing more than a high school education. FACT: the simpler a system is, and the fewer
formulas it has, the more likely it is going to perform in the future as it has in the past. So don't give much credence to all those fancy, high-
tech sounding terms used to describe how "sophisticated" or advanced a system is that it can fly a space rocket, on the contrary - run
- Easy to trade: obviously this helps. What kind of data does the system use? Is it intraday or end of day? Do you have to watch it during
the day, which is not practical for most folks. Of course, you can always turn it over to a broker to run and trade the system for you, but
even then, the easier it is for the broker the fewer the mistakes he or she will make, which like it or not do show up on your account. A
good system should also minimize slippage. Shorter term systems, especially day-trading ones can be enormously impacted if they suffer
even the slightest slippage. Those systems assume good order fills and thus look great on paper, but once they are traded live and begin
to suffer from just a few ticks of slippage per trade, there goes all that fantastic performance out the window! A good system should be
able to sustain significant slippage and still produce good results over the long term.
- Commission & Slippage Deduction: any system whose performance track record does not include a generous deduction on a per
trade per contract basis for slippage and broker commission is simply not living in reality. It is amusing to see systems, most notably day
trading ones that factor in only a $5 or $10 deduction for commission & slippage. When this value is increased to just $20 it then wipes
away a significant portion of the systems profitability. Make sure that more than enough is deducted on a per trade per contract basis for
commission & slippage without destroying profits. We use $50 for single contract examples, and $100 on a per contract basis for multi-
contract, position sizing / money management examples.
- Totally Symmetric Trading Logic: no bias towards long or short trades and can go short just as easily as long. It is amazing to see that
systems are still being designed to trade in just one direction. The argument is that the market they trade has mostly gone only one way in
the past few years - usually up. What happens if that changes? What happens if that changes and what was thought to go only one way
suddenly changes and goes the opposite way? The so called geniuses in the sophisticated financial industry got it wrong since amazingly
it even happened in the Real Estate market!!
- Tested and Verified by Futures Truth, an independent third party entity dedicated to testing and tracking trading systems. If a
developer has refused to submit their system to a respected institution like Futures Truth for an independent, un-biassed third party
evaluation it leads to concern. Some developers claim that they do not like to do this because it frequently requires that Futures Truth
have access to the system's source code. All that can be said is that based on our experience of more than a decade with Futures Truth,
to our knowledge not once have they revealed a system's logic or source code to anyone without prior authorization from the developer.
We recommend you obtain a private opinion letter on Andromeda & Pegasus from Futures Truth, along with their own up to date test
results on our systems. Futures Truth can be reached via telephone at 1 (828) 697-0273 (U.S.).
- Cost of the System: we left this for last since this is the only item on this list which should be irrelevant - the price tag of the system.
Nevertheless, as human beings it is in our nature to worry about how much we are paying for something that we are purchasing and is it
worth it. Keep in mind that what you pay for a system is insignificant compared to how much you can win or lose by trading the system. If a
system costs only $100 but you end up losing $30 thousand before you quit in disgust, your real cost was $30,100. Going for cheap can
backfire. On the other hand, an expensive system does not imply that it is a better system. We have seen systems that are distributed
completely for free - like tutorial sample systems that are included in $25 trading text books or ship as tutorial examples of trading
platforms that have handily outperform the most expensive systems. Price has no relationship to quality when it comes to trading systems -
absolutely none. Don't concern yourself with the price tag of a system, either way, be it cheap or expensive, and focus on all the other
items listed above. In any case, Andromeda & Pegasus are competitively priced and fall in the middle of the typical price ranges of
systems that are sold to the public.