Reading Assignment: Common Backtesting Mistakes

TA & Algo Trading Course
#341
  1. Dangerous is curve-fitting your strategy in the past events. Past is not Future. Over optimizing might lead to wrong trades.
  2. Around 10 years. (9-11)
  3. Complexity >> curve-fitting
1 Like
#342

1. What is so dangerous about over-optimization?

Over-optimization leads to curve fitting. It’s when algorithm that worked for back-tested historic dataset gives a false sense of predictability in future data. It’s impossible to tell the price behavior at any given time and, thus, a trading strategy in the past will not necessarily be applicable at another time frame, subject to varying circumstances (market volatility, interest rate differentials, macroeconomic variables) we cannot try to manipulate in the hope of expecting the same result. What may seem doable in theory will, in reality, most likely end us up executing the wrong future trades.

2. How long should a testing period be if you are serious about building a profitable trading strategy?

Testing and simulations should be carried out over as long a trading period as possible. In the article, processing data from over 9-11 years would be ideal as it ensures a large amount of market conditions in place, making a curve-fitted algorithm very difficult.

This presents a challenge though for altcoins (incl. BTC), which haven’t been around that long and, thus, begs the question: Could one also do this testing relative to the length of time a cryptocurrency has been around?

3. Why should you avoid asymmetric trading signals?

This would be tantamount to over-optimization of previous dataset: creating two separate sets of criteria for entering and exiting the market for long and short trades add too much complexity, which automatically increases the strategy’s degrees of freedom, making it excessively prone to curve-fitted solutions.

1 Like
#343

1. What is so dangerous about over-optimization?
Can cause curve-fitting, means the trading strategy works only with past data and not in future.

2. How long should a testing period be if you are serious about building a profitable trading strategy?
Around 10 years.

3. Why should you avoid asymmetric trading signals?
Might lead to the fact that the potential risk is not proportional to the potential reward.

1 Like
#344
  1. The high likelihood of curve fitting; tailoring your algorithm for a specific strategy rather than the focus being on the overall profitability for an extended period of time.
  2. 9 - 11 years
  3. Different strategies for long positions vs short positions increase the likelihood of not buying and selling when it’s most optimal.
1 Like
#345

  1. You can have a curve-fitting strategy. Meaning that your strategy was based on past data, and structured to it almost as if in real time. You need to keep strategy simple and elegant as to keep from having it become curve-fitting.

  2. 9-11 years.

  3. Giving different parameters to your strategy will most likely lead to your strategy becoming curve-fitted.

1 Like
#346
  1. Over-optimization easily leads to curve-fitting.
  2. According to the article 9-11 years is a good test period.
  3. Asymmetric trading signals based on past data doesn’t take into account future macro economic variables like interest rate and economic cycles.
1 Like
#347

  1. The dangers of over-optimization or curve fitting is that the strategy/algorithm will be applied to past historical data as opposed to current or future data.

  2. According to the blog post, it is recommended that the testing period should cover at least 9 years (or longer) in order to maximize the amount of data used.

  3. Asymmetric trading signals may lead to curve fitting, and may increase the chances of making mistakes.

1 Like
#348

Over optimization leads to curved fitted systems that are relying on past data. They may not preform so well on future market conditions.

A period of 5 years minim, longer if possible.

Asymmetric signals lead to another way in which curve fitting can develop. We can not tell what new data will require, making past data inaccurate.

1 Like
#349

  1. What is so dangerous about over-optimization?

    • Curve-fitting; which is alining your model with the curve to an extend where it simply follows the price movement but cannot be adequatly used to anticipate future prices.

  2. How long should a testing period be if you are serious about building a profitable trading strategy?

    • 9-11 years

  3. Why should you avoid asymmetric trading signals?

    • Using assymetric systems gives the model more freedom and makes it prone to curve-fitting. Different development of up- and downward trends are often based on increased or decreased interest or macro economics which are not technical factors and can therefore not be factored into the model.
1 Like
#350

  1. Over-optimisation tends to lead to an increased chance of a strategy being curve-fitted. I.e. adjusted to be more successful based on past data.

  2. Ideally a testing period should be 9 - 10 years of price data.

  3. They increase the solutions degree of freedom and make it excessively prone to being curve-fitted.

#351

  1. The fact that it can lead to “curve-fitting”, which is only optimised for past data. And will only show good results based on previous datasets, not necessarily current or future behaviour and changes in the market.

  2. 9-11 years, and if it yields profitable results over a ten year period then the chance of curve-fitting is greatly reduced because the system has limited degrees of freedom to artificially “fit” all those different market conditions.

  3. Adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions.

1 Like
#352

  1. Over-optimization can lead to curve fitting. Your system will perform great on the past data, but not so great with future data.

  2. Optimizations should be carried out for long periods of time, ideally 9-11 years of data should be used for the process in order to ensure that a large amount of market conditions become available. If a simple strategy yields profitable results across a ten year period then the probability of curve fitting is greatly reduced as the system has limited degrees of freedom to artificially “fit” all those different market conditions.

  3. Adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions.

1 Like
#353

  1. What is so dangerous about over-optimization?
    A strategy can be overly optimized to fit with past data which may not fit with future trading.

  2. How long should a testing period be if you are serious about building a profitable trading strategy?
    Ideally long test periods of 9-11 years which will greatly reduce the probability of curve fitting.

  3. Why should you avoid asymmetric trading signals?
    Because adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitting solutions.

1 Like
#354
  1. You can start to change your strategy based on past events and getting maximum gains on it.
    2.There should be 9 to 11 years of data
  2. There is no consistency to asymmetrical signals. Hi risk
1 Like
#355
  1. What is so dangerous about over-optimization? Over optimization can produce strategies with astonishing results that are not achievable. This can occur when short time frames are used with insufficient historical data.
  2. How long should a testing period be if you are serious about building a profitable trading strategy? Testing should be over a period of 9-11, that time period allow for the usage of different market conditions, thereby reducing the risk of cure-fitting.
  3. Why should you avoid asymmetric trading signals? Asymmetric trade signals are related to macro-economic variables that changes with economic cycles and are not reliable indicators.
2 Likes
#356
  1. Curve Fitting
  2. 9-11 years of data
  3. Past data up & down trends develop under many variables, so cannot be guaranteed to continue the same in the future. Adding sperate criteria increases the strategy’s degree of freedom and makes it excessively prone to curve fitted solutions.
1 Like
#357

What is so dangerous about over-optimization?
Crossing the divide of optimisation toward cure fitting, making the strategy react perfectly to past price action while cancelling out the unique properties it (originally) had in store for future price action.
How long should a testing period be if you are serious about building a profitable trading strategy?
5 years
Why should you avoid asymmetric trading signals?
If you discriminate opposing options in the strategy (like short and long), it is excessively prone to curve fitting.

2 Likes
#358

  1. What is so dangerous about over-optimization?
    Over-optimzation can lead to curve fitting, a common pitfall that fails to successfully predict future occurrences due to reliance on past data.

  2. How long should a testing period be if you are serious about building a profitable trading strategy?
    The standard for out-sample testing is 1-2 years to pit the strategy against draw downs and profits in new market conditions. If the test shows a loss >2x of previous max draw downs, the strategy is curve fitted.

  3. Why should you avoid asymmetric trading signals?
    Standard symmetric systems do not account for macro economic variables. The conditions inherently make it highly prone to curve fitting, since reoccurrence of variables is unlikely.

2 Likes
#359
  1. Over-optimization is dangerous because it can lead you to a trading strategy that applies in the past but it can prove unreliable in the present.
  2. Back-testing should be carried out for long time frames, 9 to 11 years, according to the article .
  3. Because the market conditions are continuously changing.
2 Likes
#360
  1. Curve-fitting. An endless effort to match all the results based on all the historical data.
  2. 9-11 years of data should be used to test numerous and various market conditions.
  3. Asymmetric signals increase complexity. Complexity, in turn, results in curve-fitting.
2 Likes