Reading Assignment: Common Backtesting Mistakes

[Shawn_Wollenberg]

1. What is so dangerous about over-optimization?
   Focus too much on past data where future data could be a little different.
2. How long should a testing period be if you are serious about building a profitable trading strategy?
   9 to 11 years
3. Why should you avoid asymmetric trading signals?
   Outside variables (like interest rates) can cause changes in these patterns that might have worked successfully with past data.

[vero]

1. What is so dangerous about over-optimization?

Developing strategies that fir past data so well that they do not perform well in the future.

2. How long should a testing period be if you are serious about building a profitable trading strategy?

Ideally 9 to 11 years

3. Why should you avoid asymmetric trading signals?

Because it makes the strategy more prone to curve-fitting. In reality, a tendency to asymmetric trading signals in the past cannot be guaranteed in the future.

[Biggiedagoe]

• What is so dangerous about over-optimization?

Past optimization cant promise results in the future

• How long should a testing period be if you are serious about building a profitable trading strategy

From the start of bitcoin till today

• Why should you avoid asymmetric trading signals?

It will lead to curve fitting as the asymmetric signals is work for the past data but we cannot sure about whether it works for the future.

[PhilD99]

1. Over-optimization leads to “curve-fitting” in which the trading strategy is adapted to market conditions of the past which are likely to change in the future, thus rendering the strategy inaccurate going forward.
2. According to the article, a minimum of 10 years data should be tested; however, this would be difficult with bitcoin, so perhaps 5 years in this case.
3. Asymmetric trading signals refers to different signals for long and short trades. This increases the strategy’s degrees of freedom, leading to a greater risk of curve-fitting.

[Rick_Pearce]

Over-optimization can lead to, to much much complexity, trying to fit your program to historical data. Which may not preform so well in the future.

9 to 11 years would be good, enough time to process a large amount of historical data.

Asymmetric trading can lead to a strategy’s freedom, making it open for curve fitting. Keep it simple.

[jacobo]

1. The fact that optimizing to a certain degree is very beneficial for better results, but it becomes dangerous when the optimization goal then gets morphed into the perfect version within optimization only for a specified block of time, and is no longer able to be applied to any other time period within the instrument to be traded.
2. It should be between 9-11 years of the instruments activity history to ensure that the code and methodology is as bulletproof as it can be for the future trading to be done; the enxtended ammount of data that is backtested will reveal wether the code had been “curve-fitted” and actually illedgible to use as a tool for a trading strategy.
3. Asymmetric trading signal code will lead to “curve-fitting” instead of sticking to the initial idea behind the strategy because te past can help predict the future, but it doesnt mean that the future will replicate the past to the same extents of movements wether positive or negative.

[Aida1]

1. What is so dangerous about over-optimization?
   it produces meaningless results and exploitation of backtesting interpolation errors.

2. How long should a testing period be if you are serious about building a profitable trading strategy?
   9-11years according to the article

3. Why should you avoid asymmetric trading signals?
   Adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions.

[Chris83]

1. You run the risk of optimizing your trading strategy to fit only the past data, curve fitting.
2. Several years to get a more accurate picture.
3. Because asymmetric conditions cannot be guaranteed in the future. That is why it is more reliable to make your strategy based on symmetric conditions.

[loso]

1. What is dangerous about over-optimization is that is may create a strategy that creates astonishing results however is specifically tailored to a data set rather than ensuring a strategy that creates optimal probabilities for events going forward.
2. The testing period should be long. The article refers to 9-11 years so that all market conditions are fitted in to the range. However in crypto that is too long. Perhaps a 1-2 year time frame with testing crypto trading strategies can implement different environments.
3. You should avoid asymmetric trading systems because it allows for more degrees of freedom for the model. It is much better to do a concise system. Asymmetric signals leads to curve fitting.

[Slavy]

1. Curve fitting - we don’t want to generate trading strategies with absolutely astonishing results that will not be achievable going forward.

2. 9-11 years

3. Because of curve fitting.

[STIWRXGUY]

1. What is so dangerous about over-optimization?
   Your strategy could go towards curved fitting. You can focus too much in making it profitable and forget about “surroundings”.

2. How long should a testing period be if you are serious about building a profitable trading strategy?
   According to the article, 9 to 11 years of market data should be taking into consideration for testing strategies.

3. Why should you avoid asymmetric trading signals?
   Because it makes the strategy prone to curve-fitting solutions.

[Javier_Flores]

Reading Assignment: Common Backtesting Mistakes

1. Curve_fitting

2. Long period gaps between. Historical points thru out the market.

3. Under past data up and down trends might have developed differently in currencies this cannot be guaranteed to continue in the future as these differences rely on interest rate differentials or such similar macro economic variables that inevitably change through economic cycles.

[Mathew]

1. The dangers of over-optimization is that if your design is too fitted to historical data, then it may not be effective if the current market changes slightly and you need to adapt.

2. A testing period should use 9-11 years of historical data to account for different markets

3. Assymetrical designs should be avoided incase of changes in the markets, to keep your program balanced.

[Edvins]

	• What is so dangerous about over-optimization?
		§ we don’t want to generate trading strategies with absolutely astonishing results that will not be achievable going forward.
	• How long should a testing period be if you are serious about building a profitable trading strategy?
		§ Optimizations should be carried out for long periods of time, ideally 9-11 years of data should be used for the process in order to ensure that a large amount of market conditions become available. If a simple strategy yields profitable results across a ten year period then the probability of curve fitting is greatly reduced as the system has limited degrees of freedom to artificially “fit” all those different market conditions.
	• Why should you avoid asymmetric trading signals?
		§ Although it is true that under past data up and down trends might have developed differently in currencies this cannot be guaranteed to continue in the future as these differences rely on interest rate differentials or such similar macro economic variables that inevitably change through economic cycles. Adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions.

[Saula_Atama]

1. What is so dangerous about over-optimization?
   Over-optimization can lead to curve-fitting which is when you adopt a bot to past data and it becomes useless with future data
2. How long should a testing period be if you are serious about building a profitable trading strategy? more than 10 years
3. Why should you avoid asymmetric trading signals?
   increases the strategy’s degrees of freedom which makes it more prone to curve-fitting.

[Sung]

1. What is so dangerous about over-optimization?
   Over-optimization may lead to curve-fitting which returns optimal results in the past but may perform horrible in the future.

2. How long should a testing period be if you are serious about building a profitable trading strategy?
   Ideally 9-11 years of data should be used.

3. Why should you avoid asymmetric trading signals?
   It increases the risk of curve-fitting. I.e. macro economic variables may change.

[Nissan_Gamba]

1. What is so dangerous about over-optimization?
   *It can lead to curve fitting

2. How long should a testing period be if you are serious about building a profitable trading strategy?
   *9-11 years worth of data

3. Why should you avoid asymmetric trading signals?
   *It will lead to inconsistency and unreliable algorithm

[Nev]

• What is so dangerous about over-optimization?
  Rules are fit to historic set, however just because it worked for that period it may not work for the future. You fill like it’s a winner but it may not be

• How long should a testing period be if you are serious about building a profitable trading strategy?
  As long as possible, 10+ years if exist but starting with say 1year first then if satisfied, test on 5 years of data you haven’t tested on

• Why should you avoid asymmetric trading signals?
  As it makes it more complex and unable to adapt to the changing trends or larger macro movements that are coming. It basically fits it to history and makes it more rigid

[Cestrada]

1. Curve fitting
2. 2 years
3. Asymmetric trading signals should be avoided because they increase the strategy’s degrees of freedom making it prone to curve fitting.

[Ren]

1. curve fitting, generating a perfect trading strategy with amazing results but becomes useless moving forward.

2. 9 to 11 years

3. End up being the same in previous market cycles which is curve fitting. Something we should avoid.