The Motley Fool Discussion Boards
Investing/Strategies / Mechanical Investing
|Subject: Re: Blending at a Whole New Level||Date: 7/24/2008 6:16 PM|
|Author: mungofitch||Number: 211475 of 258147|
This flashes my curve fitting warning light.
I agree that this process adds a bit more room for curve fitting, but there aren't
really a lot of degrees of freedom, since this is (as I understand it)
entirely a step-forward test. At each month you're using only screen
performance data for periods prior to that date. This doesn't increase
the degree to which the blend is overfit, because it isn't an
optimization on the whole test era. (two master blends, one bear and
one bull, created with the optimizer and hindsight would definitely have that problem).
Instead, subdividing the past and doing separate bull and bear blends
is really reducing the statistical support for the blends which are
created at each trading period. They will fit their smaller training
sets better, but that doesn't matter, as the results don't use those
numbers, they use the results in the period after that training.
You would expect that to reduce their predictive power. But, in this
case, it seems there was still enough information to create useful
blends for each case, as the fits were predictive enough to give good results.
The main degrees of freedom rest only in (a) the choice and
tuning of the bull/bear signal, and (b) the sort metric used to pick
the screens at each period, which implicitly includes things like the
lookback period for the screen results, the screen depth to use, and the
number of screens to use. That doesn't really seem like a whole lot to me.
To me it seems that by far the biggest issue is that when you're looking
at "only" the results up to 1998, you're using a screen whose