Reading Assignment: Common Backtesting Mistakes

TA & Algo Trading Course
#401

1-What is so dangerous about over-optimization?
The dangerous about over-optimization is ending up curve fitting and compromise the future results.

2-How long should a testing period be if you are serious about building a profitable trading strategy?
Ideally about 9-11 years of data.

3-Why should you avoid asymmetric trading signals?
Because automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions.

1 Like
#402
  • What is so dangerous about over-optimization?
    Can lead to curve fitting where you tune your strategy to fit a specific dataset but losing any value as a predictive tool for other datasets.
  • How long should a testing period be if you are serious about building a profitable trading strategy?
    9-11 years of data.
  • Why should you avoid asymmetric trading signals?
    Results in more degrees of freedom thus making it more prone to curve-fitted solutions.
2 Likes
#403

  1. What is so dangerous about over-optimization?
    The strategy would be optimised for past data, rather than based on principles. Which could lead to coding in outliers.

  2. How long should a testing period be if you are serious about building a profitable trading strategy?
    ideally 9-11 years of data.

  3. Why should you avoid asymmetric trading signals?
    It increases the degrees of freedom and more likely to be a curve fitted solution.

1 Like
#404
  1. the danger is to falsely project past results to future results
    2.10 years
    3.to avoid increasing the degree of freedom and therefore the likelihood of curve fitting which would lead to over optimisation of previous data sets
2 Likes
#405
  1. What is so dangerous about over-optimization?
    The strategy can be complicated and the results can not be profitable.
  2. How long should a testing period be if you are serious about building a profitable trading strategy?
    The longer the period the better the results, a back test period can be for a couple of years, more results can be seeing in a 10 year period.
  3. Why should you avoid asymmetric trading signals?
    It increases the strategy’s freedom and will lead to curve fitting.
2 Likes
#406

1)One of the main problems of performing optimizations is the dreaded curve fitting, if it is adjusted too closely to past events it may not be suitable for future markets.
2) around 10 years
3)Asymmetric trading signals will increase the degree of freedom which will easily allow for curve fitting.

2 Likes
#407
  1. It can dramatically affect the performance of a trading strategy, because the fact that a given system was able to exploit a market inefficiency in the past does not guarantee that the inefficiency will be present in the future.
  2. 9-11 years of data
  3. Adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions.
2 Likes
#408

1. What is so dangerous about over-optimization?
“Over-optimization” creates “curve-fitted results”, meaning that the results look amazing for past market data, but once the algorithms are applied to current data, the results aren’t good because they don’t perfectly match the curve-fitted algorithm.

2. How long should a testing period be if you are serious about building a profitable trading strategy?
Ideally 5+ years of testing data should be used; this can be tricky though if a chosen currency hasn’t been around that long.

3. Why should you avoid asymmetric trading signals?
Trading signals for longs and shorts should be the same (symmetric) as that keeps the algorithm less-prone to curve-fitting: “complexity is the mother of curve-fitting”.

1 Like
#409

1:) If the system is tweaked to get the best results using only the short time frame it may not work as well using future data or a longer time fram because it was tuned for the particular time it was tested.

2:) the longer time period the better. Testing in increment of 1 year up to the max available.

3:) Asymetric systems can lead to errors and conflicting signals and curve fitting

1 Like
#410

  1. What is so dangerous about over-optimization?
    Because it leads to curve fitting which will not bring the same results for past data.

  2. How long should a testing period be if you are serious about building a profitable trading strategy?
    The longest period the better, More than 10 years if possible

  3. Why should you avoid asymmetric trading signals?
    It will lead to mistake because over optimization

#411
  1. What is so dangerous about over-optimization? Past performance is not indicative of future gains
  2. How long should a testing period be if you are serious about building a profitable trading strategy? At least five years but 8+ is ideal
  3. Why should you avoid asymmetric trading signals? Because Changs in the market and changes global economics, we haven’t accounted for
1 Like
#412

  1. Over-optimisation is dangerous as it does not let the indicators do there job as they are tweaked just to fit charts being used as templates,

  2. 9-11years test period

3.asymmetric trading signals are more likely to create curve fitted solutions.

1 Like
#413

1. What is so dangerous about over-optimization?
By over-optimizing, you are tailoring your strategy to data from the past. We don’t know if the trends will continue in the future.

2. How long should a testing period be if you are serious about building a profitable trading strategy?
A testing period should be roughly 10 years to ensure all market conditions are tested. If your strategy is profitable over a large time period from the past it stands a better chance of being profitable in the future.

3. Why should you avoid asymmetric trading signals?
By using asymmetric trading signals you open yourself up to excessively curve fitting.

1 Like
#414
  1. Over-optimization can dangerously lead to an unprofitable solution for the future because it’s tailored to previous market conditions.
  2. 9- 11 years of data should be used for the process in order to ensure that a large amount of market conditions become available.
  3. Trend can not guarantee to continue in the future as differences relies on interest rate or macro economics variable that change through economic cycles .
1 Like
#415

image
image828×608 186 KB

If I have developed a strategy can I test it on the history of other assets or instruments to increase degrees of freedom and minimize curve fitting?

1 Like
#416

1.curve fitting or over optimization is dangerous because it yeilds great results in the back test but not during live trading.
2.1-2 years and tested on data that you didnt optimize to.
3. because it leads to curve fitting.

1 Like
#417
  1. What is so dangerous about over-optimization?
    can lead to curve-fitting, which is a unwanted tuning of the strategy in order to fit it into a specific data set.
  2. How long should a testing period be if you are serious about building a profitable trading strategy?
    1 or 2 year periods over a span of 10 years
  3. Why should you avoid asymmetric trading signals?
    it increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions.
1 Like
#418
  1. When you try to repeat a past event, normaly you always miss something. The past events can give you a indicator for the present, but the market reaction is never a copy paste of the past.
  2. Testing period should be as long as we ca affort to be and should catch as many diferent reactions of the market as possible.
  3. A state of asymmetric information exists if one party has information advantage that the other lacks. This is said to cause market failure. This way the correct price cannot be set according to the law of supply and demand .
1 Like
#419

Hi everyone,

  1. It leads to curve-fitting : your solution will fit the data very well and achieve good performance for the past but will not be achievable in the future.
  2. 9-11 years
  3. It increases the strategy’s degrees of freedom and makes it prone to curve-fitted solutions.
1 Like
#420
  1. What is so dangerous about over-optimization?

You can loose a lot of money thinking your strategy is going to work well when it won’t. You’ll also waste time and over-complicate things.

  1. How long should a testing period be if you are serious about building a profitable trading strategy?

The article states ideally 9-11 years of data. Not too sure how many tokens have that length of trading history.

  1. Why should you avoid asymmetric trading signals?

Adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions. Adhere to the KISS principle (Keep It Simple Stupid).

2 Likes