[QUOTE=pdp-brugge;23510]Trev,
I do intent to review all the criteria I use on a regular base. The next time I will redo the complete study will probably be end of September. Now that my back test environment is setup (the programming is done) all it takes is changing the parameters and let the system run a couple of thousands simulations. When I say redo, I mean also evaluating if other XLX models become more interesting then the ones I have been using.
Of course I use data-mining techniques (I have been a software-developer of analytical accounting software in the past). I have been careful not to over optimize the system. Because my window of data is rather small (January 2010 up until now) the statistical evidence is also rather small. I hope over time this will improve simply because my database will grow. Because my equity curve has already had a serious decline, I have been quit honest. The equity curve has been going up again the last 4 weeks.
PdP[/QUOTE]
Hi PdP (and Pascal and Billy if interested),
I noticed this and would be interested in your thoughts and do you agree, strongly disagree with the writer :-
=========================================================
MarketSci Blog
--------------------------------------------------------------------------------
Backtests Ain’t Worth the Pixels They’re Written On
Posted: 29 Aug 2012 09:55 PM PDT
The most common feedback (by far) I’ve received about our proprietary strategies over the last 6+ years have been requests for backtested results.
My response has always been that I don’t release backtested results for our own strategies…ever.
Backtests are perfectly fine for testing, sharing, and discussing ideas; that’s more or less the only thing we do here on this humble blog.
But backtests should never be used to judge the efficacy of a black box you’re considering committing hard earned funds to.
I’m cynical. I’ve seen way too many examples of folks who sold based on backtests, only to fall woefully short in real-time trading. I’m sure you could recount your own share of horror stories.
The only thing that really matters in judging a strategy is actual, real-time, verifiable trading results. Everything else is window dressing.
If I could somehow guarantee that folks would see backtests for what they’re really worth: best guesstimates, guilty of the same cognitive biases that we ALL face and inherently curve-fit, I would release them in a heartbeat.
But I can’t guarantee that.
I know that readers’ eyes get wide when they see a sexy equity curve because even though I know I shouldn’t, mine do too.
So sorry, but I can’t in good conscience share backtests. Rest assured that they would be sexy, and your eyes would go wide, but that, like all backtests, they wouldn’t be worth the pixels they were written on..........
[url]http://marketsci.wordpress.com/2012/08/30/backtests-aint-worth-the-pixels-theyre-written-on/[/url]
========================================
Trev