Reading Assignment: Common Backtesting Mistakes

TA & Algo Trading Course
#281
  1. Curve fitting is the thing we should be worried about the most. If to over-optimaze strategy, it might show excellent results with past-data, but it will not fit with future trends.
  2. Ideally test periods should be 9-11 years with time frames greater than 30 minutes.
  3. It will increase degrees of freedom which can result in curve fitting with past-data and will not fit in future trends.
1 Like
#282

  1. What is so dangerous about over-optimization?
    Over-optimization creates curve fitting because fine-tuning a strategy to a specific data set doesn’t optimize different scenarios.

  2. How long should a testing period be if you are serious about building a profitable trading strategy?
    9 to 11 years using time frames no less than 30 minutes.

  3. Why should you avoid asymmetric trading signals?
    Because it increases the strategy freedom and makes it prone to curve-fitted solutions.

2 Likes
#283

  1. What is so dangerous about over-optimization? - Curve fitting = any given “curve” or data set can be accounted for by a given mathemtical function of arbitrary complexity.

  2. How long should a testing period be if you are serious about building a profitable trading strategy? - “9-11 years in order to ensure that a large amount of market conditions become available.”

  3. Why should you avoid asymmetric trading signals? - Past data cannot guarantee the future so by using asymmetric signals makes you more prone to “curve fitting.”

2 Likes
#284

1- It can lead to curve fitting, which is the inability to work with actual market behavior because the strategy was based on limited past events.
2-Testing periods should be as long as 10 years.
3-It increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions.

1 Like
#285
  1. You can find a mathematical function which can predict with absolute accuracy the past, with no ability to predict the future.
  2. 9-11 years
  3. Makes the strategy excessively prone to curve-fitted solutions
1 Like
#286
  1. Over-optimisation is dangerous because by tailoring a strategy to a specific historical dataset severely limits the possibilities of this outcome happening again. It’s also known as over-fitting.
  2. I believe is this subjective, particularly when we back-testing with Cryptocurrency because the market is constantly changing. A year should be sufficient and more for conventional markets.
  3. You should avoid asymmetric trading signals because in order to have sustainable trading strategies your risk reward ratio must be balanced, otherwise you’re trading a fine line with gambling.
1 Like
#287

  1. What is so dangerous about over-optimization?
    The main goal is to develop a strategy that will make good returns over a long time period. With over-optimization you might get a perfect strategy, but only for a small time frame. And if the prices in the future don’t beheave, they most likely won’t, like in the over-optimized strategy, it would be useless, even though it was showing good results in the backtest.

  2. How long should a testing period be if you are serious about building a profitable trading strategy?
    In the article it is stated that the testing period should be 9-11 years. But when the article was released in 2010, you were not able to trade BTC, so it was written for the traditional financial market. Depending on how much details the strategy has, I would say 6 months to 5 years (one complete bear/bull cylce)

    1. Why should you avoid asymmetric trading signals?
      Adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions.
1 Like
#288

  1. Because you are getting data from the past and likely stocks/cryptocurrency will act differently in the future

  2. 9-11 years that it for traditional market. But for example we think BTC I would say 4-6 years is more than enough because of BTC cycles

  3. Because ut increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions

1 Like
#289
  1. What is so dangerous about over-optimization? making a strategy that works only on an historical segment of chart and not in the future.
  2. How long should a testing period be if you are serious about building a profitable trading strategy? approximately 10 years
  3. Why should you avoid asymmetric trading signals? because there could be external factors that influence a specific time frame that could have greater/lesser effect on the shorter or longer time frame.
1 Like
#290

  1. Over-optimisation of a trading strategy by adjusting its parameters according to specific past market data is dangerous because by doing so we may create a strategy that is perfect for performing well in a specific scenario, but may not be well rounded to perform well going forward.

  2. A trading strategy should be tested over a longer period of time, ideally 9-11 years. This is so it can be tested across as many different market conditions as possible so that it is battle tested.

  3. You should avoid asymmetric trading signals because these signals may differ in the future due to macroeconomic factors and therefore may be invalidated.

1 Like
#291
  1. Keep it simple. The more complex your algorythm is the more problematic it will become. Curve Fitting will happen.
  2. Minimum 5 years; best 9-12 Years - the longer the testing period /simulation the more possible market conditions will be covered. Less than 5 years could be very problematic.
  3. It has you ending up with a curve fitted programm as it takes away simplicity of the code.
1 Like
#292

[quote=“filip, post:1, topic:7688”]

  • What is so dangerous about over-optimization?
    Achieving astonishing past results will not be achievable going forwards.
  • How long should a testing period be if you are serious about building a profitable trading strategy?
    9-11 years of data should be used for the process
  • Why should you avoid asymmetric trading signals?
    will make is excessively prone to curve fitting
1 Like
#293
  1. it can lead to curve fitting, which will most likely lead to wrong future signals
  2. 9-11 years
  3. it can also lead to curve fitting
1 Like
#294
  1. There is a danger of fitting a strategy to the sample data and have a false idea that the strategy works while it wouldn’t work in another timeframe (ex: the future)
  2. Around 10 years
  3. Even if the longs and shorts might behave differently in a given period, they depend on external macroeconomic factors that are somewhat unpredictable. Having asymmetric trading signals increases the risk of curve-fitting.
1 Like
#295

  1. Making mistakes due to a non clear indications, which might turns to losses in the future.

  2. Ten years.

  3. Might lead to mistakes due to asymmetric signals.

1 Like
#296

  1. You can historical data and variables to predict a trend and indication but not suitable for present /future market or data.

  2. As many years back to give the max amount of data.

  3. Having asymmetric parameters makes the model excessively prone to curve-fitting solutions

1 Like
#297
  1. It is dangerous to over optimize a trading strategy as this can lead to curve fitting and results that are only true given exact past market situations which might not be true again in the future. This can lead to poor results.
  2. 5 years!! Hmm is this true in crypto, I’m not sure.
  3. Adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions
1 Like
#298
  1. Over-optimization runs the risk of curve fitting which bases your model on what has exactly happened in the past to predict what the future will look like. This gives you a false sense of risk management.
  2. 9-11 years of data but 1-2 year test samples
  3. It greatly increases the chances of curve-fitting.
1 Like
#299

What is so dangerous about over-optimization?
– That you focus only on the past data and try to optimze this with the example data sheet.

How long should a testing period be if you are serious about building a profitable trading strategy?
– It should include bull runs and bear markets but over all you can us the timeframe from 9-11 years

Why should you avoid asymmetric trading signals?
– Asymetric trading signal are outside the norm and you can only use it from PAST data.
Could that work, yes. But the chance to fail is high. Who has know that TESLA will allow payments with BITCOIN…

1 Like
#300

1.What is so dangerous about over-optimization?
Over Optimization can lead to a strategy becoming curve-fitted

2.How long should a testing period be if you are serious about building a profitable trading strategy?
Testing periods should be 9-11 years for profitable trading strategies and should include out-sampling to ensure curve fitting does not occur.

3.Why should you avoid asymmetric trading signals?
They are inaccurate at predicting future movement. More data is needed.

1 Like