Algorithmic Trading Group Forum
Trading Strategies >> Development >> Evaluation of Strategies

Message started by Madhur on 12/05/08 at 08:40:28

Title: Evaluation of Strategies
Post by Madhur on 12/05/08 at 08:40:28

Before developing strategies, one needs to have a method to evaluate not just the returns or RAR performance of a strategy, but to estimate the probability that the candidate strategy is not showing superior performance out of sheer chance.
Moreover, if there is any optimisation of parameters or selection from a large number of strategies, then there is a need to establish that the candidate would work across a large time span (20 years +). If substantial amounts of data is used for such optimisation/selection, then how can one do an ex-poste test over such large time spans.
Any ideas?

Title: Re: Evaluation of Strategies
Post by Yury R on 12/05/08 at 14:29:16

May be I am saying something naive and stupid, but you trading system has to rely on some properties of the underlying price process, right?

Then, perhaps, instead of using rules of thumb "one size fits all" approach for backtesting it is not right. Can the underlying process be modelled somehow and then the model properties are validated independently in different ways?

Of course, you don't know what the underlying process is, but it can be assumed to be something and compared with reality statistically or it can be reverse-engineered from the trading rules.

Validation of the model can be done in number of ways. It's properties can be attributed to some observation about human behaviour or physical reality or whatever. Literature can have some hints that these processes do take place in economy.

The model will have some unknown parameters which can be estimated irrespective of any particular trading system. Some intervals of uncertainty can be postulated and trading system can be tested with different values from these intervals. Or even with stochastic values. Then you get an idea about statistical properties of the model, argue about work of chance and drop some rules that have no relation to the observed model etc...

Also you'll get a separate tool to watch the market in real time in order to catch regime change, i.e. not when the trading system performs badly but when the price changed its behaviour in the aspect that you look at it with you model.

As I said, it may be very immature to hope for this methodology to work, but that is my current understanding of things.

Title: Re: Evaluation of Strategies
Post by sayandeep on 04/13/09 at 12:51:42

A strategy that will work for 20+ years! I think that is a little bit ambitious :)

The best strategy is the one which none has thought about at that point of time and hence will make money. But with so many smart people around soon everyone will flock towards that strategy to make money. And that is precisely when a new strategy will need to be developed.


Title: Re: Evaluation of Strategies
Post by guru on 05/25/10 at 17:55:34

The three most important things , I want to look at when I am evaluating a strategy are:

- Win Ratio
- CAGR (Compound Annual Growth Rate)
- Number of days a trade is held

I have done some backtesting on candlestick chart patterns that indicate that the probability of trend reversal (win ratio) is around 46-52% for SP500 stocks over a period of 15 years. This may / may not be a good starting point for building a system. I am not sure and am still thinking about it

What do you think?

Title: Re: Evaluation of Strategies
Post by Lawrence on 11/20/10 at 03:14:53

One can consider looking here:

Here is what the page states:

The Trading Systems Project is a response to many FAQ questions about mechanical systems.

TSP is an opportunity for readers of FAQ to participate in the design, verification, testing and implementation of an actual trading system, from the ground up.

To participate in the Trading Systems Project, just follow along with the evolution of the project and volunteer when you feel you have something to contribute or can cross-check the results.

Title: Re: Evaluation of Strategies
Post by cjforex on 07/26/11 at 09:45:43

I like to first see in sample and out of sample show similar behaviour. Then it goes into limited forward live testing. Slowly portfolio optimization algos feed it more money based on its performance compared to its brethren.


Algorithmic Trading Group Forum » Powered by YaBB 2.2.2!
YaBB © 2000-2008. All Rights Reserved.