May be I am saying something naive and stupid, but you trading system has to rely on some properties of the underlying price process, right?

Then, perhaps, instead of using rules of thumb "one size fits all" approach for backtesting it is not right. Can the underlying process be modelled somehow and then the model properties are validated independently in different ways?

Of course, you don't know what the underlying process is, but it can be assumed to be something and compared with reality statistically or it can be reverse-engineered from the trading rules.

Validation of the model can be done in number of ways. It's properties can be attributed to some observation about human behaviour or physical reality or whatever. Literature can have some hints that these processes do take place in economy.

The model will have some unknown parameters which can be estimated irrespective of any particular trading system. Some intervals of uncertainty can be postulated and trading system can be tested with different values from these intervals. Or even with stochastic values. Then you get an idea about statistical properties of the model, argue about work of chance and drop some rules that have no relation to the observed model etc...

Also you'll get a separate tool to watch the market in real time in order to catch regime change, i.e. not when the trading system performs badly but when the price changed its behaviour in the aspect that you look at it with you model.

As I said, it may be very immature to hope for this methodology to work, but that is my current understanding of things.

The three most important things , I want to look at when I am evaluating a strategy are:

- Win Ratio
- CAGR (Compound Annual Growth Rate)
- Number of days a trade is held

I have done some backtesting on candlestick chart patterns that indicate that the probability of trend reversal (win ratio) is around 46-52% for SP500 stocks over a period of 15 years. This may / may not be a good starting point for building a system. I am not sure and am still thinking about it

What do you think?

I like to first see in sample and out of sample show similar behaviour. Then it goes into limited forward live testing. Slowly portfolio optimization algos feed it more money based on its performance compared to its brethren.

Cjforex