The Motley Fool Discussion Boards
Investing/Strategies / Mechanical Investing
|Subject: Re: Blending at a Whole New Level||Date: 7/24/2008 8:06 PM|
|Author: elann||Number: 211476 of 254160|
This flashes my curve fitting warning light.
I agree that this process adds a bit more room for curve fitting, but there aren't
really a lot of degrees of freedom, since this is (as I understand it)
entirely a step-forward test. At each month you're using only screen
performance data for periods prior to that date.
What you're describing is the lack of a look-ahead bias, i.e. crystal ball, which is true.
What I'm describing is a different issue. Let's say you have two screens A and B, and A performs slightly better over the whole test period. Now you perform some random process to split the test period into alternating subperiods (simulating a timing signal). You call the two sub-series "bull" and "bear". Then you test both screen A and B in the bull and bear periods separately, and pick out the one that performed better in each period. You have created two new opportunities to outperform A alone - one where A is "bull" and B is "bear", and the other where they are reversed.
Do you think this is totally legitimate? Now what if I revealed to you that the original A and B were two instances of the same screen run with different starting dates within the month. Clearly, therefore, the difference in performance between A and B was due to chance. And the timing exercise only gave luck an opportunity to work harder.
This is of course not exactly what Zee did. It's done for illustration only, and I hope nobody wastes time telling me how the metaphor is different from reality.
Maybe Zee is onto something, but as you say, I'm dubious.
|Copyright 1996-2014 trademark and the "Fool" logo is a trademark of The Motley Fool, Inc. Contact Us|