Reading Assignment: Common Backtesting Mistakes

TA & Algo Trading Course
#221
  • What is so dangerous about over-optimization?

Over-optimization might result in a model that just works for the period of time where the model is backtesting, reducing the probability of working on any other time period let alone future periods.

  • How long should a testing period be if you are serious about building a profitable trading strategy?

About 9-11 yeras, but in cryptomarkets I am not sure. Maybe 2-3 years.

  • Why should you avoid asymmetric trading signals?

Asymmerric trading signals increase the bias of the model so that it might not work under general conditions of the market.

1 Like
#222

[quote=“filip, post:1, topic:7688”]

  • What is so dangerous about over-optimization?
    Over optimization can lead to “curve-fitting”, which is the unwanted tuning of the strategy to fit specific past data.

  • How long should a testing period be if you are serious about building a profitable trading strategy?

  • Time frames should be greater than 30 minutes

  • Ideally 9-11 years of data should be used for the process in order to ensure that a large amount of market conditions become available.

  • Why should you avoid asymmetric trading signals?

  • Adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitting solutions.

1 Like
#223

1.The dangers of over-optimization is curve fitting, trying to make it fit too much wont work on future data, the less complexity and parameters the least likely to be curve fitting
2.Atleast a Year or above , the longer the better, eg 10 Years would be a reliable time to test the conditions of the market
3.points you in the direction of curve fitting which will not work

1 Like
#224

  1. It is dangerous as it can throw your system in the complete different direction right off the jump. If a system is too over-optimized, you give no flexibility to your trades which are in the FUTURE and undecided. Any wiggle room is essential.

  2. The sweet spot according to this article, and possibly mathematically is 10 years of historical data (9-11). More or less is prone curve fitting

  3. Avoiding asymmetric trading signals helps not congest data for the system to show results. It is advised to create a separate system for a different trading signal.

1 Like
#225
  1. The increased probability that with a slight change in market condintions our strategy would be unprofitable or ,worst case scenario , disastrous
  2. According to the article the ideal testing period would be 9/11 years and trade timeframes no lower than 30 minutes
  3. We should avoid asymmetric trading signals to prevent our strategy to be curve-fitted that is precisely what we want to steer clear of at all costs
1 Like
#226
  1. It may give rise to curve fitting.
  2. The longer better, at least 5 years.
  3. Asymmetric, again may curve fit, making no guarantee will perform better.
1 Like
#227
  1. It can deliver Curve Fitting, where the data provided fits past trading so well it cannot make any
    future trade predictions, so does not preform nearly as well as expected.
  2. 9 to 11 years at least so as to capture all market conditions possible.
  3. It can lead to you matching up with pervious market cycles.
1 Like
#228

Over-optimization is to produce a trading strategy with absolutely astonishing results that will not be achievable going forward.

About 10 years of data should be used to ensure accuracy

Asymmetric indicators are prone to curve fitting

1 Like
#229

  1. What is so dangerous about over-optimization?
    It creates a curve-fitting phenomenon making the selected criteria fit old data to perfection but making unreliable simulations for future market moves.

  2. How long should a testing period be if you are serious about building a profitable trading strategy?
    About worth a decade.

  3. Why should you avoid asymmetric trading signals?
    These are different criteria for entering/exiting a position. Doing this increases the strategy’s degrees of freedom which in turn increase the chances of curve fitting.

1 Like
#230

  1. What is so dangerous about over-optimization?
    Optimization is the process by which a given system has its parameters adjusted to give better performance in the past data. Over-optimization is when you modify the parameters to fit the past data perfectly, to the point that it is not feasible to work on new data. This is dangerous because you will have a model that you think it works, but it won’t.
    One thing to avoid is curve-fitting, meaning you can find a mathematical function which can predict with absolute accuracy all the items of a data set.
    Avoid model complexity.
    ​​​​​​​

  2. How long should a testing period be if you are serious about building a profitable trading strategy?
    Optimizations should be carried out for long periods of time, ideally 9-11 years of data should be used for the process in order to ensure that a large amount of market conditions become available

  3. Why should you avoid asymmetric trading signals?
    Because even if it worked with the past data, this cannot be guaranteed to continue in the future, because of macro economic variables that inevitably change through economic cycles

1 Like
#231

  1. It risks giving you false results.

  2. Ideally 10, but since crypto is so new it might be better to use one year.

  3. It makes the strategy too fine-grained. There is also no guarantee that the perceived difference between e.g. up and down trends will remain in the future.

1 Like
#232

1. What is so dangerous about over-optimization?

You run the risk of curve-fitting your trading strategy to the past data events to the point that your model can have detrimental consequences on current and future data where your model may simply perform very inaccurately.

2. How long should a testing period be if you are serious about building a profitable trading strategy

  • Should build for timeframes greater than 30 minutes
  • Should test across 9-11 years of market data to check for consistent results across various market situations

3. Why should you avoid asymmetric trading signals?

Adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions.

1 Like
#233

1- Can lead to curve-fitting, the strategy will be tunned to past data and can not be good for future data
2- 9-10 years
3- Adding different parameters for long and short trades increases the degree of freedom and makes it prone to curve-fitting solutions

1 Like
#234
  1. What is so dangerous about over-optimization?

It will limit the freedom of the strategy and will make the strategy more curve-fitted.

  1. How long should a testing period be if you are serious about building a profitable trading strategy?

9-11 years

  1. Why should you avoid asymmetric trading signals?

it automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions.

1 Like
#235
  1. curve fitting. The strategy is suited perfectly for the past event, but in this way just a little change in the future could destroy the strategy
  2. 9-11 years of data are perfect
    3)adding separate criteria for longs and shorts increase the strategy’s degree of freedom.
1 Like
#236

  1. Over-optimization leads to curve fitting where your strategy works only on the initial data you were backtesting on

  2. 9-11 years of data

  3. Because giving different parameters for entering/exiting positions will give your strategy more degrees of freedom which often leads to curve fitted results

1 Like
#237

  1. What is so dangerous about over-optimization?
    The curve fitting

  2. How long should a testing period be if you are serious about building a profitable trading strategy?
    From 9 to 11 years

  3. Why should you avoid asymmetric trading signals?
    Because of the change of interest rates, and the macro economic variables

1 Like
#238

Lesson 8: Reading Assignment: Common backtesting mistakes

  1. What is so dangerous about over-optimization?

Over optimization leads to increased and excessive correlation of data resulting in curve fitting of the data.

  1. How long should a testing period be if you are serious about building a profitable trading strategy?

The longer the better but ideally nine to eleven years.

  1. Why should you avoid asymmetric trading signals?

It cannot be relied upon that these same signals in currencies will happen in the future, because these differences rely on interest rate differentials or other similar macro economic variables that inevitably change through economic cycles, thus rendering your data optimization useless due to curve fitting.

1 Like
#239
  1. What is so dangerous about over-optimization?
  2. How long should a testing period be if you are serious about building a profitable trading strategy?
  3. Why should you avoid asymmetric trading signals?

1- Over optimization is unrealistic and plays like a Monday morning QB, or, as we say in Argentina, you use Monday’s newspaper, and since you never have the complete picture in real time investing, the system will be flawed.

2- Around 10 years of data is the desirable time span.

3- Separate criteria makes the system prone to curve fitting since you will be accommodating the variables to the desired outcome, rather than analyzing the data itself.

1 Like
#240

Over-optimization – if systems are overly complex, programs lend themselves to Curve fitting. So get it simple over long periods of time (T > 6 yrs) and backtest. Remember, using separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom, and makes it excessively prone to curve-fitted solutions.