Reading Assignment: Common Backtesting Mistakes

TA & Algo Trading Course
#261

1- It can be over adjusted to past data, and may not work for the future, we also need to have in mind the macro panorama for the traded asset.
2- Accoding to the article from 9 to 11 months.
3- Asymmetric information can lead to adverse selection, incomplete markets and is a type of market failure, you end up with matching previous market cycles.

1 Like
#262
  1. What is so dangerous about over-optimization?

The results are so tailored for the past that they will not have the same results in the future and not be relevant.

  1. How long should a testing period be if you are serious about building a profitable trading strategy?

9 - 11 years of data

  1. Why should you avoid asymmetric trading signals?

Assymetric trade signals make it excessively prone to curve-fitted solutions. The differences in the past rely on interest rate differentials or similar macro economic variables that will inevitably change throuhgout the economic cycle.

1 Like
#263

  1. Over optimization can lead to a curve fitting strategy. That means we optimized our strategy to perfectly fit the past data, but will not perform as well with future data set.

  2. More than 5 years and preferable 9-11 years.

  3. Adding asymmetric trading signals leads to complexity which can lead to curve fitting.

1 Like
#264
  1. Strategies will take to much freedom when there are many parameters to define, or divergencies in the input data, It will make it a “subjective” strategy hard to follow
  2. According to the article about 10 years with a two years out-sample test.
  3. Because the asymmetry responds often to a short time periods, so it makes an unlikely profitable strategy in the future
1 Like
#265
  1. Overoptimization may lead to losses since high risks of curve fitting past data is incurred.
  2. 9-10 years, as it is stated in the blogpost
  3. Assymetric trading signals should be avoided fro they may rely on incorrect past data interpretation as they are based on variables, that change during econimic cycles
1 Like
#266
  1. Strategy that highly fits the past data, might not be general a general method that could also fit the future situation on the market. Strategy should have a satisfactory performance based on the past data and a high possibility to repeatedly perform well in the future.
  2. 9-11 years of data to ensure that strategy includes different market situations.
  3. If according to past data, up and down trends developed differently, there is no guarantee that this can continue in the future. Adding additional variable to fit long and short positions differently can increase the uncertainty of our strategy.
1 Like
#267

  1. Over-optimization may lead to curve fitting. Therefore, it will align with past data very well, but may not work for future models. The key is to keep it simple.

  2. 9-11 years is the ideal timeframe to analyze different possibilities in market cycles. In crypto, it may not be possible, but as long as possible should help. Analyzing different markets such as the stock market doesn’t give a good indication as to changes in the crypto market.

  3. Assymetric trade signals may overcomplicating the strategy and lead to curve fitting.

1 Like
#268

** 1. What is so dangerous about over-optimization?** It can result in curve fitting. The goal is to produce systems that achieve good performance in the past with the highest possible guarantee that that performance will be repeated in the future it becomes vital to take steps in order to ensure that optimization does not deliver curve-fitting.

** 2. How long should a testing period be if you are serious about building a profitable trading strategy?** The article advises a period of between 9-11 years however in the Crypto world this would realistically be around 4 years due to the infancy of the currency.

** 3. Why should you avoid asymmetric trading signals?** Adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions.

1 Like
#269
  1. What is so dangerous about over-optimization?
  • Your trading strategy will have no effect because thinks are not repeat in exactly the same way.
  1. How long should a testing period be if you are serious about building a profitable trading strategy?
  • as long as possible. min. 1 year
  1. Why should you avoid asymmetric trading signals?
  • because you focus to much on maybe one wrong signal.
1 Like
#270

  1. What is so dangerous about over-optimization?
    At the worst the strategy will be fitted to the curve the asset to be traded did produce in the past. This would cause that the achieved result can not be reproduced int the future as the strategy fails.

  2. How long should a testing period be if you are serious about building a profitable trading strategy?
    Ideally 9-11 years. A Strategy which does prove profitable over a period of 10 years is likely to do so in the future as well. In crypto such long time frames are rare. Best would be about a 4 year period, 3 years for back testing and 1 year (forward testing) to confirm the performance of the strategy. If less data is available I would always recommend to use a 3 to 1 ratio back testing and forward testing (confirmation).

  3. Why should you avoid asymmetric trading signals?
    Asymmetric signal selection is prone to curve fitting. Asymmetric signals may depend on the environment which is devoted to change.

1 Like
#271
  1. What is so dangerous about over-optimization?
    Over-optimization or fitting the curve too close to the data has the benefit of including more of the other independently acting systems inherent to the overall model, but it includes the danger of tracking the behavior of sub-systems which, like the eddy currents along the rivers’ edge, do not have such a simple, predictable action, as the main body of the river.

As it would require the identification of a sub-system or more, the mathematics of the study of the system would involve the isolation of the sub-system(s) into planes of behavior which create borders between the original plane of behavior and interfering planes, such planes which would require the resolving of variables which move as contra-variants to what the observer would expect as co-variants.

Each set of data describes the statistical or static behavior as if all variables and systems of sets of variables acting as sub-systems of the original set which may not be true under conditions of increasing testing

  1. How long should a testing period be if you are serious about building a profitable trading strategy?

The writer of the article suggests a 9-11 days testing period based on a model of no less the 30 minute events, or so-called time frame. This is an interesting expectation, for it defines the order of the complexity of the system being studied and the rigor by which such complexity requires for describing the continuity of such a systems’ behavior.

  1. Why should you avoid asymmetric trading signals?

“asymmetric trading signals” are to be avoided as it invites the mixing of degrees of the action of variables within the originating orders of the systems by which the variables are studied. The variables within the study are defined as containing the highest order of the action of the variable within the conditions of the study, the main function existing as causation in the matter. However, it may be effect of something else unknown and unseen, and thus, it may be a dependent variable, dependent upon the something unknown at the present time of the study.

For the research minded individual, I believe there is a middle ground between optimization and asymmetric trading signals, which is to measure the lowest degrees of change within the highest order of behavior to be observed.

2 Likes
#272

What is so dangerous about over-optimization?
Making it so it works on the previous data but not on the new data.

How long should a testing period be if you are serious about building a profitable trading strategy?
9-11 years of data.

Why should you avoid asymmetric trading signals?
Adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions.

1 Like
#273
  1. What is so dangerous about over-optimization?

By over-optimizing a model for the same test period you run the risk of optimizing the model for historical events which might very well not be representing future events.

  1. How long should a testing period be if you are serious about building a profitable trading strategy?

according to the article 9-11 years, >30min timeframes

  1. Why should you avoid asymmetric trading signals?

Adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions.

1 Like
#274

  1. Over-optimization runs the risk of curve-fitting, trying to fit an algorithm to past data.

  2. Approximately 10 years is a good time period for back-testing.

  3. Asymmetric trading is prone to curve-fitting solutions.

1 Like
#275
  1. What is so dangerous about over-optimization? If a strategy is too well fitted for past data, it’s likely to not be profitable under future circumstances.
  2. How long should a testing period be if you are serious about building a profitable trading strategy? More than 5 years, preferably 9-11 years of data.
  3. Why should you avoid asymmetric trading signals? Adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions
1 Like
#276
  1. Over-optimization decreases the odds of a trading strategy working in the future
  2. 9-11 years
  3. They rely too much on macroeconomic variables that will change in the future
1 Like
#277
  1. What is so dangerous about over-optimization?

You can do many little mistakes such as using highly correlated procedures or ignoring the surroundings of your intended results. That can lead you to curve-fitting.

  1. How long should a testing period be if you are serious about building a profitable trading strategy?

It should be very long, ideally 9 to 11 years.

  1. Why should you avoid asymmetric trading signals?

That makes you more prone to curve-fitting.

1 Like
#278

  1. if your strategy works for past data analysis the dangerous part could be to think that it will definitely work for the future.

  2. 10 years

  3. It may be prove to curve fitting as the asymmetry may work for particular market cycle with which it is applied yet not as effective when applied on different market cycle or asset.

1 Like
#279

  1. What is so dangerous about over-optimization?
    Over-optimization can result in “curve-fitting” or tuning the strategy to fit specific past data that will not be replicable in high probability

  2. How long should a testing period be if you are serious about building a profitable trading strategy?
    A testing period should be ideally 9-11 years to ensure that a large amount of market conditions are included

  3. Why should you avoid asymmetric trading signals?
    Adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitting solutions.

2 Likes
#280
  1. good for past, not for future trading
  2. 10 years
  3. makes it more complex
1 Like