No. of Recommendations: 2
I know it is uncomfortable to leave in the October 2008 outlier, but then
you should also remove the October 1987 outlier for the futures model.
But that does not make sense either since you are removing dis-similar outliers.

Usually I leave in all the bad news.
In this case it doesn't matter since the crash of '87 was not at month end based on any conceivable tuning.

Leaving in one extraordinary bit of good news has the effect of making
the system look a whole lot better and more trustworthy.
I would rather err on the side of distrusting the system than trusting it too much.
Heaven knows that most predictive models for market direction are weaker than they appear in backtest.
Realistically speaking, there is no way in hell I would have been long at the end of October 2008.
There's what makes sense, and what is possible for a mere human to carry out.

I do the flip side for 1987 when testing other timing systems: I leave
the crash in, but force the model to be wrong during it to see what happens.
A timing system whose rule is "be long all the time except October 1987"
by chance is remarkably beguiling based on risk and return figures.

Print the post  


What was Your Dumbest Investment?
Share it with us -- and learn from others' stories of flubs.
When Life Gives You Lemons
We all have had hardships and made poor decisions. The important thing is how we respond and grow. Read the story of a Fool who started from nothing, and looks to gain everything.
Contact Us
Contact Customer Service and other Fool departments here.
Work for Fools?
Winner of the Washingtonian great places to work, and Glassdoor #1 Company to Work For 2015! Have access to all of TMF's online and email products for FREE, and be paid for your contributions to TMF! Click the link and start your Fool career.