Hi guys, hi from Andrea Unger! Today, I’d like to talk about Walk Forward Analysis, a very interesting topic that, sometimes, causes some confusion in people. So, I’d like to explain what it is and when we can use it.
People often ask me what I think about Walk Forward Analysis and if I use it. I think it is a good analysis method. Nevertheless, I don’t use it. The reason is quite simple and depends on the specific way in which I build my trading systems.
I must admit many people have a somehow confused idea of Walk Forward Analysis or Optimization. In fact, they consider it a simple validation of a system. According to them, “forward” refers to seeing how the system behaves once you stop developing it and look at what it does in real time. This is out-of-sample analysis; it’s fine, but is not the Walk Forward Analysis.
The Walk Forward Analysis is an approach where you develop a system with parameters and then you optimize these parameters over a certain period of time.
Suppose we work on a 4-week period. Once we find which parameters led to the best performance during the last 4 weeks, considering that you might believe that what worked well for the last 4 weeks is supposed to continue working quite well in the coming week as well, you put this combination of parameters at work in the coming week.
So, you test what happened over the last 4 weeks and then use the optimum of these 4 weeks on the following one week.
Once this week is over, you focus over the last 4 weeks again, which means you forget about the first week and take the last four. You run a new optimization and find a new combination of parameters. In the ideal case, and quite seldom, they are the same as the previous period. After finding the new parameters, you use them in a sixth week and then go on like this.
This is one method. However, there is also another way that consists in optimizing on the whole history. So, you always add one week to the history and use the optimum of the complete history you have at your disposal in the coming week.
This is not my favourite approach. When you come to the end of a history of, say, 200 weeks, you might have some trouble finding what works best in the last period, which is the very idea that stands behind this analysis method.
What works well now, should also work well in the next couple of days or week.
I think the real advantage of using the Walk Forward Analysis to develop systems is that it allows you to create a large out-of-sample history, as you have many weeks when you actually test the results in out of sample.
So, you also have an indicator of how good and robust your system is, and if it keeps on doing well once it goes live. You have a wide out-of-sample period that is not the classic last two years or first two years, but that considers the whole historical data you have.
This is certainly very good, so, why don’t I use it? The reason lays in the way I develop my trading systems, which creates some difficulties in identifying what parameters to optimize.
If you develop a system using indicators, you have plenty of parameters to choose from and can obviously do a good job. However, if you have watched other contents on this blog, you probably know that I don’t use indicators in my development.
I use chart patterns, instead. They are listed in some function to use in the development phase. However, they only give me an idea of how the market moves. For instance, I have a pattern that tells me if there has been a period of indecision or decision, a yes/no thing, a 50-50. I have a parameter here (0.5), but it would make no sense to optimize it from, say, 0.45 to 0.55 with 0.01 steps.
It would be pure overfitting, no longer seeking for the best parameter. The idea I have is “yes, it’s an indecision phase, no is a strong decision phase”, that’s it.
So how can I really use this? You may suggest that I play with a different list of all the patterns and look for the best one. The point is that this pattern is a go/don’t-go thing. If the pattern is true you trade, if the pattern is false you don’t trade.
Again, looking for the best pattern probably means cherry-picking the best trades, but how can I know that this pattern will occur again in the week that we analyse afterward?
As you can see, the way I develop systems creates a very different structure, so that, in this structure of mine, it’s very hard to apply the Walk Forward Analysis as it is conceived in the original form.
That’s the reason why I don’t use it. I’ve also talked to Kevin Davey – a Walk Forward Analysis expert who knows about my approach and development techniques – and he came to my same conclusion: it’s very hard to apply the Walk Forward Analysis concepts to my way of developing systems.
This, again, does not mean that the Walk Forward Analysis is wrong. Actually, I really appreciate it and think it’s a good tool.
If in your developing way it works, perfect, use it! And if you know how to make it work for my approach, I will listen to you very carefully.
If you don’t want to use it because you don’t believe in it it’s fine as well, provided you make some money with your trading systems.
The Walk Forward Analysis is a further application of using numbers. This is algo trading and is what I recommend to everybody.
That’s it, see you next time!
Ciao from Andrea Unger!
