We'll help you map out a plan to fix the problems in your trading and get you to the next level. Answer a few questions on our application and then choose a time that works for you.
BOOK YOUR FREE STRATEGY SESSION NOW >>Hi guys, hi from Andrea Unger! Today, we talk about backtesting and, more in particular, about how long the history for the backtest should be. Which data should we take into account when we evaluate the performances of a strategy?
The key word when we talk about how to use historical data in backtesting is “depends”. It depends on the market and on the conditions.
Let’s see what we can do when we backtest strategies on stocks and indexes.
Generally speaking, the best thing to do is to take into account as many data as possible, in order to cover different market conditions. So, our historical data should cover bullish, bearish and choppy phases, as well as wild crashes. The more phases we have, the easier to get an idea about how the market reacts to our models.
The problem is that market phases don’t always show the same reaction to our approaches. So, you might think that your model is wrong or whatever, whereas, sometimes, it is the market that moves and reacts in a different way.
Think of indexes such as the e-Mini S&P 500. Generally, I say that you need a bull market in your data. Of course, we have a lot of this in our data, and we also have bad crashes and bear phases. This is true, but I can tell you that, in many cases, some approaches produce different results in the bear market of the 2001/2002 compared to the bear market of 2008 or in the crashes we saw in August 2015, February 2018 or late 2018.
So, it’s very difficult to identify a common behaviour of that market in those different conditions. So, the more we have the better. This allows us to see how stable the approach is, although understanding why is still quite difficult.
Backtest on Market Crashes
Certainly, a violent crash is different from a slow-down or prolonged down phase. Down phases are never too slow and are always a bit active.
Sitting in front of my PC screen, I finally realized why breakout strategies on the e-Mini S&P 500 seem to be more efficient after a market crash. Compared to classic countertrend models, they generally produce more satisfactory results during these phases.
The key is in the e-Mini S&P 500 order book. Normally, there can be 300, 400 or even 900 contracts at each level of bid/ask price, depending on the time of the day. After market crashes such as the one occurred in February 2018, there can be only 30 or 40.
This leads to obvious consequences. If you send a large order, say 500 contracts, and there are 900 in the order book, you don’t move the market that much. Instead, if there are only 30 contracts, you eat out different levels of bid or ask and cause the prices to move. This means you work in favour of the breakout.
So, actually, once the big players have gone out of the market, waiting for the situation to calm down, the market starts to behave and react in a completely different way. This is something we must be aware of, because it can happen. Eventually, we can choose not to use any strategy or to choose some strategies that are specifically devised for such periods.
However, keep in mind that we can never actually know how long these phases will last. It can be one week, one month, six months, whatever. So, actually, we have to be very careful and have enough data to understand this.
Whereas on indexes it’s better to have a large variety, on other markets we sometimes have to understand that there are conditions which will never come back.
For example, on commodities such as the Crude Oil, the Gold and Grains, intraday data are available starting from about 2008. When I develop on these markets, I usually start from January 1, 2010. The reason why I avoid using data from those two years, although I pay for them, is that I think they are not significant.
These markets became electronic in 2007-2008. During those first years, they were getting accustomed to the new situation. They were learning how to trade electronically, together with the pit and so on. The performances our systems make during those years cannot be trusted.
Even when those data are correct (because sometimes they are not), they should not be used. During those years, those markets went through conditions that will never come back again. Those conditions depended on a specific condition – the transition to the electronic market – which will never happen again in the same way.
So, those results are useless in order to understand the behaviour of a market. They don’t help us understand how the market will react to our systems.
On the other hand, results after those days show a different reaction to trend-following, to breakout, on many markets. In fact, there has been a tendency to lose efficiency in breakout strategies; as they gained efficiency, the markets show a stronger resistance to breakout.
Soybeans, for example, showed a very strong change in its behaviour in 2011-12, whereas the Gold future showed that change in 2018.
Sometimes, markets resume their past behaviour and go through other trend-following periods. Nevertheless, we should be aware that the trend-following feature is getting weaker on these markets.
Reading backtest results on a wider range of data can help us understand what’s going on, so we need that. Anyway, when it comes to commodities, I think we’d better start from 2010. As for index futures, it’s better to go back as far as possible, trying to understand how the market reacted to every single phenomenon, drawing comparisons and studying results very thoroughly.
When it comes to the Forex market, we should point out that, whereas currencies continued behaving the same way throughout the years, there were changes in their volatility levels. The volatility that characterized the Forex market in the older years is now much smaller. So, today it’s much harder for certain approaches that are based on that kind of volatility to make profits.
So, again, it’s better to look back and always keep in mind that overall conditions can change. Read the reports carefully and try to understand what works today, still looking back if it worked in the past.
Last 12 Months Backtest, Is It Useful?
Some traders also wonder if taking into account the last 12 months of data Is enough. If we have a strategy that has been losing money for years and made a lot of money over the last 12 months, can we consider it a good strategy? Personally, I would not put that strategy live. In fact, I think 12 months are too short a period to make any decision, especially when this period strongly differs from what was shown before.
If there was an improvement, that can be okay. However, if there is a huge or complete difference with what happened before, unless we identify a clear reason for this change, we should wait for a longer time to decide to put this live.
Perhaps those 12 months foretell how the market will behave over the next 10 years. In that case, we will lose some time waiting for further confirmation, but I’m still convinced that 12 months are not enough, unless, perhaps, you trade high frequency on tick data or so. However, that’s not my – and probably your - case, so I would not use that.
To sum it up:
I hope it helped. Write your comments on what you prefer and how long you look back when you develop your strategies.
Ciao from Andrea Unger!
We'll help you map out a plan to fix the problems in your trading and get you to the next level. Answer a few questions on our application and then choose a time that works for you.
BOOK YOUR FREE STRATEGY SESSION NOW >>Andrea Unger here and I help retail traders to improve their trading, scientifically. I went from being a cog in the machine in a multinational company to the only 4-Time World Trading Champion in a little more than 10 years.
I've been a professional trader since 2001 and in 2008 I became World Champion using just 4 automated trading systems.
In 2015 I founded Unger Academy, where I teach my method of developing effecting trading strategies: a scientific, replicable and universal method, based on numbers and statistics, not hunches, which led me and my students to become Champions again and again.
Now I'm here to help you learn how to develop your own strategies, autonomously. This channel will help you improve your trading, know the markets better, and apply the scientific method to financial markets.
Becoming a trader is harder than you think, but if you have passion, will, and sufficient capital, you'll learn how to code and develop effective strategies, manage risk, and diversify a portfolio of trading systems to greatly improve your chances of becoming successful.
