How to Avoid Overfitting in Systematic Trading

Overfitting is a systematic trader’s worst enemy. In this article, we will explain what it is, what its causes are, and how to avoid it when developing a trading system.

What Is Overfitting

"Over-fitting" means over-optimizing.

To understand what overfitting is and how to avoid it, it is essential to clearly understand "optimization".

As the name suggests, optimization is the process of improving a trading system.

In trading, optimizing a system means changing some inputs and parameters to experiment with  which inputs lead to improvements in performance.

Optimization represents one of the most important phases in the development of a trading system. Its main objective is to verify the stability and robustness of a system based on setting certain inputs.

The choice and variation of parameters, however, if done in an excessive and instrumental way, can significantly alter the performance of a strategy.

Let's take an example.

If we wanted to evaluate the speed of travel on a particularly busy road and took peak times as a reference, we would certainly get negative values.

However, if we restrict the analysis to just holidays, the data would inevitably become much more positive. The road is always the same in terms of its holes, its narrowings, and its daily flow of traffic but simply by manipulating some parameters, we have obtained the best possible result.

Overfitting happens when you force the modifiable parameters of a system with the aim of finding the combination that generates the best profits.

How Overfitting Works

Overfitting is a procedure of over-optimizing a strategy in order to obtain the combination that offers the best possible results.

The outcome of this process is an equity line that generally appears to perform well.

The problem, however, is that these performances are not the result of a solid and robust strategy but derive, as in the example of traffic, from an excessive manipulation of the parameters.

This means that these results can only be obtained in that way and in that specific time frame and are not significant for evaluating the real effectiveness and reliability of the system in the future.

In addition to the manipulation of parameters, overfitting can also involve the overly selective choice of the financial instruments where the system is applied.

For example, suppose you want to trade the stocks of the Dow Jones or the S&P500 indexes.

In these cases, the correct attitude would be to trade all the markets in the index and not limit yourself only to the stocks that showed the best performance in backtesting.

If you pick only the stocks with the best equity lines you risk producing distorted results. This can lead to results that can’t be replicated.

Sometimes it can happen that you are forced to make a selection of the instruments in a basket, for example due to budget reasons.

In these cases, to avoid falling into overfitting or overexposure, you should proceed logically, adding or eliminating the various stocks based on objective criteria such as volumes and liquidity.

In this way, it will be easier to obtain "truthful" results that have a greater chance of repeating themselves in the future.

Remember that making predictions about the profitability of an over-optimized strategy is not only wrong but highly dangerous.

The Risks of Overfitting

The main danger for an over-optimized system is that of failing to adapt to real markets, thus causing severe losses to the trader.

Generally, such losses occur as soon as the system goes live. However, systems may also perform successfully for some time before crashing.

In these cases, it is said that the system "broke".

In addition to overfitting, systems can stop working for other reasons, including running out of inefficiencies found in development.

It is the inefficiencies of the markets that allow traders to profit.

The problem is that inefficiencies can run out, both due to the behavior of other traders and to inevitable changes in the markets.

Consequently, those edges will no longer be exploitable as in the past.

So what to do when a system stops working?

In the following video, Andrea Unger explains how to insert logical rules into a system without forcing them. He also shows, through a practical example, what to do when levels of inefficiency discovered in the development phase gradually disappear from markets.

As you will see in the video, the only way to avoid overfitting and to exploit market inefficiencies for as long as possible is to constantly study and search for new trading ideas and new markets and instruments.

How to Avoid Overfitting: Possible Solutions

To avoid falling into overfitting it is important to:

• Always proceed logically and rationally, following a scientific method

• Avoid excessive filters and rules

• Optimize only the essential parameters

• Skillfully manipulate inputs during development (don't just focus on getting the best results)

By following these tips it is possible to develop systems that are truly reliable and robust.

Remember that the goal of optimization is not to find the combination of parameters that produced the best performance in the past. The goal is a system that produces stable results even out of sample.

Do you want to learn how to understand if a system is robust or not? Click here!

Conclusions

Overfitting is one of the biggest dangers a trader can fall into, especially at the beginning of their career.

It consists in the over-optimization of a system to obtain the combination of inputs that returns the best results.

However, manipulating or discretionally choosing the parameters of a trading system to achieve the highest net profit is a very serious mistake.

This system adapts perfectly to past data but is unable to be profitable out of sample.

It should be remembered that the goal of optimization should be to verify the reliability and stability of a system and its ability to be potentially profitable also in the future.

For this reason, when developing a strategy, it is essential to proceed logically, following a scientific method without seeking the combinations of inputs that produce the most beautiful equities.

Manipulating the parameters of a trading system to obtain the best net profit is an error that undermines the future reliability of the system.

Only by studying and following a method with proven results is it possible to avoid overfitting and other common trading pitfalls.