The Motley Fool Discussion Boards

Previous Page

Investing/Strategies / Mechanical Investing

URL:  http://boards.fool.com/some-of-the-independant-variables-are-highly-30535311.aspx

Subject:  Re: SoWR and New Relative Value Screen Date:  2/10/2013  6:56 PM
Author:  tetranomad Number:  241534 of 263227

Some of the independant variables are highly correlated and this will cause the tuned parameters to differ fom their expected values even to the extent of sign reversal. So the tuned parameters may be misleading outside of the model, but the descriptive model as a whole, remains valid.

I suspect you already know this, but it is worth re-emphasizing - extensive tuning to fit historical data often leads to worse results going forwards than less tuned variants. I find what Bill Eckhardt has to say on the topic (albeit wrt futures trading) useful:

from http://www.futuresmag.com/2011/03/01/william-eckhardt-the-ma...

FM: Talk about the battle between optimization and curve fitting.

BE: By trying to improve your system you can make it worse. You can over-fit to past data or maybe just do something that is statistically invalid. There is an idea, though it is not universally subscribed to, that you should not optimize your systems. That you should just figure out what are reasonable numbers and go with that. I don’t believe in that; we optimize all the time, but there is some truth to it in the sense that if you over-fit, you are going to hurt yourself. Optimizing is a somewhat hazardous procedure, as is trading. And it has to be done with carefulness and deliberateness, and y