Skip to main content
No. of Recommendations: 26
Whoop8:
With regard to your third annotation, if I change the "Hold if still passing..." from step 0 to "ril.w != null" the CAGR increases dramatically from 17.8% p.a. to 25.3% p.a. Given the fact that there are significantly different annual returns in the two versions of the backtest, this suggests to me that the portfolio compositions must be different. Could you please comment on that phenomenon?

The phenomenon is called a "bug", which has now been fixed. Amazingly, this one appears to have survived for the entire life of the backtester since 2007, including a re-writing of almost all of its C++ code from 2012 to 2014. It was quite a pain to track this one down, because it only manifested when the backtester was compiled in release mode, not in debugging mode.

I've checked everything I've used the backtester for myself and I'm relieved to see that nothing appears to have been impacted by the bug. However, others will not be so lucky. I've temporarily preserved the buggy version of the backtester at http://gtr1.net/3142 so that others can assess how much the bug has cost them, and so that discrepancies can be reported. You don't need to go looking for them, because I know they exist, but for any screens that you customarily use or have used over the last thirteen years, please email me (my fool.com username at gmail dot com) the URLs for any discrepancies between the two versions of the backtester so that I can make sure they are as I would expect.

It definitely pays to report anything about the backtester's behavior that doesn't make sense, rather than just go with the URL that produces the highest CAGR. In Whoop8's case, the lower CAGR was the correct one.

Robbie Geary
Print the post Back To Top
No. of Recommendations: 15
Thanks again for the ongoing work. We don't say it enough.

Is there a set of conditions which bound the situations likely to have triggered the bug?
e.g., were all known cases of bad results specific to the "hold if still passing" section, or all "hold till..." group of options?

Jim
Print the post Back To Top
No. of Recommendations: 6
The phenomenon is called a "bug"... It was quite a pain to track this one down, because it only manifested when the backtester was compiled in release mode, not in debugging mode.

Which makes it not just a bug, but a good example of the endearingly-named Heisenbug.

- Jamie
Print the post Back To Top