Reading Assignment: Common Backtesting Mistakes

TA & Algo Trading Course
#241

Common Back testing Mistakes. Reading Assignment.

1 what is so dangerous about over-optimization is curve-fitting systems ie generating trading strategies with astonishing results that will not be achievable going forward.
The goal here is to produce systems that achieve good performance in the past with highest possible guarantees that that performance will be repeated in the future

  1. A testing period for building a very profitable strategy should be 9 - 11 years for the process. This will ensure that a large amount of data and market conditions will be available and can be used for testing.

3 The reason why you should avoid asymmetric trading signals is that adding separate criteria for longs and shorts trading, automatically, increases the strategy’s degrees of freedom and makes it excessively prone to curve - fitted solutions.

1 Like
#242
  1. What is so dangerous about over-optimization?
  • it could end up to a curve-fitting which won’t work in the future…most likely
  1. How long should a testing period be if you are serious about building a profitable trading strategy?
  • minimum of 5 years, ideally 01-11 years
  1. Why should you avoid asymmetric trading signals?
  • it could end up to a curve-fitting
1 Like
#243
  1. Over optimization may result in curve fitting which can perfectly predict the past but may wreck you since you are trying to predict the future.
  2. A testing period should be around 11 years.
  3. risks curve fitting
1 Like
#244
  1. when you over optimize you start to curve fit your strategie and it only becomes good in the past
  2. it should be between 9 and 11 years
  3. it invites complexity and tends to turn to specific data sets
1 Like
#245

  1. curve-fitting. Whenever you give a strategy enough degrees of freedom an optimization will yield curve fitted results. The less complexity and less parameters available within a given strategy the less probable it is that it will ever be curve fitted as systems that don’t have complex criteria tend to be unable to “fit” to the data if a true inefficiency is not present.

  2. Long testing periods. Optimizations should be carried out for long periods of time, ideally 9-11 years of data should be used for the process in order to ensure that a large amount of market conditions become available.

  3. system symmetric Although it is true that under past data up and down trends might have developed differently in currencies this cannot be guaranteed to continue in the future as these differences rely on interest rate differentials or such similar macro economic variables that inevitably change through economic cycles.

1 Like
#246

  1. What is so dangerous about over-optimization?
    It may lead to Curve - fitting, which is a strategy that gives a false sense of predicting future data based on a specific dataset from the past.

  2. How long should a testing period be if you are serious about building a profitable trading strategy?
    9 to 11 years.

  3. Why should you avoid asymmetric trading signals?
    Asymmetric trading signals add complexity to the strategy and may lead to curve fitting.

2 Likes
#247
  1. curve fitting is the most dangerous result of over-optimization, it´s like making a piece of a furniture like a closet or something , separately and try to fit it in the whole thing , it doesn’t fit
  2. a very long time test period
  3. because: “Adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions.”
1 Like
#248

  1. Curve fitting in a way that perfectly shows the past but there is high possibility to not work predicting the future.

  2. Close to 9 - 11 years

  3. It can affect curve fitting

1 Like
#249
  1. What is so dangerous about over-optimization?
    Curve fitting is the danger of over optimization. You may great fantastic results on your test data but you could be devastatingly wrong in the real world.
  2. How long should a testing period be if you are serious about building a profitable trading strategy?
    You should test your model over 9 to 11 years worth of data to see how it responds to all sorts of changes in market conditions.
  3. Why should you avoid asymmetric trading signals?
    Adding asymmetric trading signals leads to complexity which can lead to curve fitting
2 Likes
#250
  1. Curve fitting- to overoptimize your system in such a way that it will only work in past data, where simulations have been done.
  2. 9-11 years of data plus 1 year or 2 out-sample
  3. the asymmetry is chosen based on past data, which is not a guarantee this will work in the future as well
1 Like
#251

  • What is so dangerous about over-optimization?
    Over optimization will cause a trading bot to rely to much on specific historical data

  • How long should a testing period be if you are serious about building a profitable trading strategy?
    Approximately 8 years. Ideal is 10 years

  • Why should you avoid asymmetric trading signals?
    Because you want to minimize the risk of curve fitting

1 Like
#252

  1. Over optimization may lead to curve fitting which is based solely on past data and may not hold true for future data.

  2. A testing period should be from 9-11 years in length if you are serious about building a profitable trading strategy.

  3. Asymmetric trading signals can lead to curve fitting because it’s based on past data and the future data may not hold true.

1 Like
#253

  1. The fact that we can adapt our strategy to the data of the past so accurate that it will not be able to work in the future in a profitable way.

  2. We should use a time frame from 9 to 11 years and never less than 5 years.

  3. Because it gives an extra degree of freedom to the system that will make the system more likely to be curve-fitted.

1 Like
#254
  1. Over optimization leads to tuning the strategy to fit the specific data from the past know as “curve fitting”
  2. Ideally 9-11 years so I would figure that the longer your period the more accurate you’ll be
  3. Adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions.
1 Like
#255

What is so dangerous about over-optimization?
The problem applies to trading because the fact that a given system was able to exploit a market inefficiency in the past does not guarantee that the inefficiency will be present in the future. Usually optimization is great to curve-fit trading systems because what an optimization does is merely to “adjust function parameters” to find a mathematically sound answer to the problem. The better and tighter the optimization, the more curve-fitted the system will become, this is a reason why neural networks – which are excellent at optimization- tend to fail in successful trading systems as they always curve-fit their data excessively.
So what is curve-fitting anyway ? Simply explained, the term is derived from the fact that any given “curve” or data set can be accounted for by a given mathematical function of arbitrary complexity. That is, you can always find a mathematical function which can predict with absolute accuracy all the items of a data set. However, the function may have absolutely no predictive power.

How long should a testing period be if you are serious about building a profitable trading strategy?
Optimizations should be carried out for long periods of time, ideally 9-11 years of data should be used for the process in order to ensure that a large amount of market conditions become available.
So if you want to optimize your system and avoid curve fitting, use a period of at least five years. Using a smaller period will most likely “fit” your strategy to very specific market conditions and will make it unable to perform correctly as the market changes.

Why should you avoid asymmetric trading signals?
Adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions.

1 Like
#256
  1. What is so dangerous about over-optimization?

we don’t want to generate trading strategies with absolutely astonishing backtested results that will not be achievable going forward.

  1. How long should a testing period be if you are serious about building a profitable trading strategy?

Ideally 9-11 years and at least 5 years

  1. Why should you avoid asymmetric trading signals?

Adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions as using an indicator cross at 20 for long entries but 15 for shorts

1 Like
#257
  1. the strategy will be perfect for past data but not the future trading
  2. 9-11 years
  3. Because the can lead to curve-fitting in the long run.
1 Like
#258
  • What is so dangerous about over-optimization?
    Over-optimization can give rise to curve fitting. The more complexity that’s in a system the more likely it will have some level of curve fitting
  • How long should a testing period be if you are serious about building a profitable trading strategy?
    , Time frames should be more than 30 mins and ideally 9-11 years of data should be used for the process
  • Why should you avoid asymmetric trading signals?
    It adds unneeded complexity to the system which can increase the changes of curve fitting
1 Like
#259

  1. What is so dangerous about over-optimization?
    Curve fitting is the risk.

  2. How long should a testing period be if you are serious about building a profitable trading strategy?
    For a period of at least one year, during which we are improving our program. After that, we should test the program for different periods equivalent to 9-11 years, to check if the program is profitable on different charts.

  3. Why should you avoid asymmetric trading signals?
    asymmetric trading signals can lead to curve fitting.

1 Like
#260
  1. The more a strategy is optimised, the more likely it is that the strategy has adapted to specific conditions in the past data which will not occur in the future.
  2. The data set should be around 10 years to ensure that you have covered a variety of market conditions.
  3. Adding different criteria for longs and shorts makes the strategy more prone to curve-fitting due to the influence of macro events on past data sets.