UnThreaded | Threaded | Whole Thread (34) | Ignore Thread Prev | Next
Author: Zeelotes Big red star, 1000 posts Feste Award Nominee! Old School Fool Add to my Favorite Fools Ignore this person (you won't see their posts anymore) Number: of 252185  
Subject: The Best Measure for the Best Blend Date: 11/16/2007 12:01 AM
Post New | Post Reply | Reply Later | Create Poll . Report this Post | Recommend it!
Recommendations: 75
What is the absolute best measure for finding the best blend?

If it isn't already, this should be the key question in everyone's mind -- at least, everyone who employs a blend for their MI investing. I've done hours and hours of testing over the past year or so to answer this question, compiling tests that include hundreds of millions of data points. At the very core, each test is trying to determine which measure is the most predictive of future return.

How did you set up the tests?

Well, I've run hundreds and hundreds of tests, but for this post, I'll be sharing based on using ranks 1-10 of each screen with a backtest history from 1969 to present. From these backtests I find five screens to invest in at the beginning of each year from 1989 using a descending sort of each of the measures. The # of lookback years to consider is varied from 1 to 29 in steps of two for a total of 15 lookbacks for each measure. I then take the results from this test and find the Minimum, Maximum, Median and Average value for each measure. I would consider the median and maximum to be key, but I also pay attention to the degree of deviation from max to min. The measure you actually use will also be used with a fixed number of years lookback so the maximum value is probably the one you will end up with, but we want to consider all to determine how robust the measure is. A vast span between min and max reflects a measure that probably just landed on such a high value by luck.

In summary, the period tested is 1989 to 11/9/2007 and the # of screens held is five holding ranks 1-10 for a total hold of 50 stocks. The idea here is not to necessarily hold 50 stocks, but to use this as a basis for finding the best measure. I've actually tested the exact same thing as this using ranks 1-4 and the results are not that much different, although the values are all higher.

All of the tables that follow are sorted by the Median column with the best values at the top.

What is the best measure for finding the highest future CAGR? In other words, which measure is most predictive of future CAGR?

A measure that is not talked about on this board comes out on top -- Jensen. The next one down is Alpha, also not often discussed as far as using it to predict future CAGR. The one that comes out on the absolute bottom is Sharpe/GSD recently suggested by StevnFool.

When looking at these returns, keep in mind that using the Solver to optimize and select a blend at the end of each year produced these results:
A Sharpe Optimized Strategy
CAGR around 26
Sharpe around 1.20
This then is the benchmark that we are comparing to.
RANKS 1-10                                 
CAGR Minimum Maximum Median Average
Jensen 29.33% 39.17% 37.50% 35.77%
Alpha 28.33% 38.34% 37.48% 35.85%
UP 27.96% 37.54% 36.95% 34.57%
Treynor 31.75% 37.08% 35.48% 35.13%
GSD 27.47% 35.18% 34.95% 33.58%
DD 27.86% 35.72% 34.93% 33.22%
Trough # 26.33% 34.49% 33.06% 31.83%
UI 27.08% 34.28% 31.78% 31.07%
CAGR/UPI 26.04% 36.02% 29.75% 29.85%
UPI 22.69% 28.36% 27.62% 26.67%
Sharpe 26.51% 30.91% 27.26% 27.70%
CAGR/UI 22.54% 27.56% 26.90% 26.07%
Sortino 24.29% 30.38% 26.37% 26.34%
UPR 4.88% 27.72% 26.25% 24.34%
Beta 20.83% 27.16% 24.81% 24.44%
GSD Ratio 20.67% 30.58% 23.57% 24.32%
CAGR/GSD 21.68% 24.03% 22.18% 22.42%
Correlation 20.35% 23.53% 21.80% 21.88%
Sharpe/GSD 20.14% 23.83% 21.78% 21.61%

But our goal is risk-adjusted return, not raw return, so what is the best measure for finding the highest Sharpe Ratio?

Well, you'll be surprised to learn that what is at the absolute bottom of the list above, comes out at the absolute top of the list for risk-adjusted return -- StevnFool's Sharpe/GSD. BTW -- I just added this measure to my backtester after he suggested it yesterday. As a result I had to run a special test just for that indicator and add it to all my other tests.
  Sharpe     Minimum  Maximum  Median  Average
Sharpe/GSD 1.25 1.67 1.62 1.58
Treynor 1.38 1.62 1.54 1.52
UPR 0.16 1.62 1.47 1.33
Sortino 1.02 1.45 1.37 1.36
CAGR/GSD 0.98 1.58 1.34 1.33
Sharpe 1.03 1.45 1.33 1.34
UPI 0.86 1.26 1.24 1.16
CAGR/UI 0.86 1.26 1.18 1.16
Alpha 0.91 1.22 1.18 1.15
Jensen 0.94 1.22 1.17 1.13
UI 1.03 1.21 1.17 1.13
CAGR/UPI 0.90 1.25 1.14 1.12
UP 0.81 1.13 1.11 1.03
GSD 0.84 1.09 1.09 1.04
DD 0.87 1.08 1.07 1.01
GSD Ratio 0.89 1.37 1.05 1.05
Trough # 0.76 1.06 1.01 0.97
Correlation 0.79 1.02 0.93 0.93
Beta 0.63 0.78 0.73 0.72

What measure comes out on top for Ulcer Index?

Once again we find that Sharpe/GSD comes out on top. We also see that although Jensen is great at finding the best screens to invest in at certain times, it does so at the expensive of a much higher Ulcer Index. If your goal is to find screens with just amazing forward returns then Jensen may be for you, but be forewarned, it will not be a smooth ride to the top.
    UI       Minimum  Maximum  Median  Average
Sharpe/GSD 3.40% 3.64% 3.62% 3.57%
CAGR/GSD 3.43% 10.89% 4.10% 4.54%
Sortino 4.68% 9.16% 4.93% 5.46%
UPR 0.00% 9.37% 5.19% 5.06%
Sharpe 5.33% 9.82% 5.64% 6.19%
CAGR/UI 6.38% 12.26% 6.49% 7.51%
UPI 6.48% 12.40% 6.52% 7.55%
Treynor 6.16% 7.60% 6.61% 6.64%
GSD Ratio 5.71% 11.42% 8.10% 8.63%
Correlation 6.92% 21.86% 8.42% 9.03%
UI 8.97% 12.63% 9.23% 9.77%
CAGR/UPI 8.37% 12.92% 9.27% 9.48%
Jensen 15.27% 19.46% 15.55% 16.29%
Alpha 15.32% 21.41% 15.62% 16.54%
Trough # 17.55% 29.73% 17.81% 21.16%
GSD 19.02% 31.45% 19.18% 20.80%
UP 21.62% 31.55% 21.67% 23.70%
DD 21.91% 31.81% 21.92% 23.95%
Beta 29.10% 35.59% 30.96% 31.99%

Which measure chooses screens with a high win ratio?

In other words, which of the measures tends to choose screens that consistently produce a positive return each year? Note again which measure comes out on top! Indeed, this simple little idea that StevnFool has come up with has produced some amazing results.
 Win Ratio   Minimum  Maximum  Median  Average
Sharpe/GSD 89% 92% 91% 91%
UPR 0% 89% 88% 82%
CAGR/GSD 86% 91% 88% 88%
Sortino 83% 88% 86% 87%
Sharpe 83% 87% 85% 85%
CAGR/UPI 77% 85% 85% 83%
Treynor 81% 89% 84% 85%
UI 77% 87% 84% 83%
GSD Ratio 77% 86% 83% 82%
Jensen 79% 83% 82% 81%
UPI 78% 85% 82% 82%
CAGR/UI 80% 85% 82% 83%
Alpha 79% 83% 81% 81%
Trough # 72% 80% 78% 77%
GSD 75% 81% 77% 77%
Correlation 72% 78% 77% 77%
DD 74% 80% 77% 77%
UP 69% 81% 76% 75%
Beta 71% 77% 74% 73%

When it comes to the maximum daily drawdown, which of the measures is able to avoid drawdowns the best?

Of course, here we are not suggesting that max drawdown is predictive of anything, but rather, evaluating how well a measure does at avoiding drawdowns. Once again we find that the winner is Sharpe/GSD.
  Drawdown   Minimum  Maximum  Median  Average
Sharpe/GSD -22% -16% -17% -17%
CAGR/GSD -43% -21% -21% -23%
Sortino -40% -22% -23% -25%
UPR -41% 0% -25% -23%
Sharpe -40% -23% -25% -28%
Treynor -33% -26% -28% -28%
UPI -45% -29% -29% -31%
CAGR/UI -47% -29% -29% -31%
CAGR/UPI -45% -31% -31% -33%
UI -41% -32% -32% -34%
Correlation -62% -31% -34% -35%
GSD Ratio -52% -25% -39% -41%
Jensen -60% -44% -44% -47%
Alpha -62% -44% -44% -48%
Trough # -72% -48% -48% -56%
GSD -75% -52% -52% -56%
DD -76% -58% -58% -62%
UP -74% -60% -60% -63%
Beta -75% -69% -71% -71%

Which of the measures you used actually produced a backtest with the lowest GSD?

You guessed it -- Sharpe/GSD comes out on top once again! In fact, this new measure comes out on top for every evaluation except CAGR. I sure do like the looks of those GSD values for the Sharpe/GSD strategy. A GSD around 10 is pretty much unheard of.
    GSD      Minimum  Maximum  Median  Average
Sharpe/GSD 9.71 13.16 10.06 10.25
CAGR/GSD 11.60 18.26 12.68 13.36
UPR 0.21 17.97 14.39 13.73
Sortino 14.41 22.09 14.92 15.82
Sharpe 16.05 22.14 16.49 17.23
CAGR/UI 18.15 23.73 18.28 19.01
Treynor 18.02 20.11 18.43 18.55
GSD Ratio 17.62 21.42 18.74 19.30
UPI 18.59 24.24 18.79 19.43
Correlation 18.60 26.04 19.53 19.98
CAGR/UPI 22.50 27.28 23.25 23.58
UI 23.13 29.35 23.50 24.39
Alpha 25.17 29.87 28.75 28.27
Jensen 27.60 29.52 29.07 28.92
GSD 29.38 34.52 29.92 30.39
DD 30.40 35.87 30.98 31.48
UP 31.12 35.82 31.19 31.99
Trough # 30.39 36.45 31.22 32.08
Beta 34.03 36.77 35.60 35.65

It seems that Sharpe does not seem to be the best measure based on your tests -- why is that?

I have no idea, but it definitely does not come out on top. In fact, here is its position for each of the evaluation tables above.
  Sharpe   Position
CAGR 11
GSD 5
Sharpe 6
UI 5
Win Ratio 5
Drawdown 5

So out of the nineteen measures that I used to sort on, Sharpe takes 11th place when it comes to CAGR, and sixth when it comes to Sharpe itself. In other words, there are much better measures to use than Sharpe itself for determining the best blend to invest in.

Remember that using the Solver to find a new blend each year resulted in a CAGR around 26 and a Sharpe around 1.20. Using the Sharpe/GSD above produces a CAGR around 1.65 and a CAGR around 24. Which would you say is better?

NOTE: I actually had no intention of sharing even this much research on this theme, but decided to open the flood gates and let the generosity flow -- being that the holidays are just around the corner and all. I hope you've found it useful, and I also hope it promotes some discussion that will be a benefit to us all.

Further Research

A very interesting bit of information that comes out of this research is the fact that some of these measures do exceedingly well at finding the best screens for when the market is bullish, and others find the best when the market is bearish. If you have even a very simple way to determine the bullish and bearish periods you'll do very well switching between the measures used. For example, when the market is bullish use the Jensen -- it found five screens for a total hold of 50 stocks that produced a return of 131% in 1991 and 180% in 1999 and close to 90% in both 2003 and 2006. Pretty amazing! Now if you use a SMA on the market and switch to Sharpe/GSD during the bearish periods you'd get 45% in 2000, 27% in 2001 and 15% in 2002. The problem with Sharpe/GSD is that it really tends to under-perform when the market is super-bullish like 1991 and 1999, but does great the rest of the time. It seems to me that a very simple method can be found that would enable anyone here to switch their sorting method to fit the market. But of course, this would be timing, and there are so many strongly opposed to that no matter how sound the evidence is supporting its benefit. Right? Even when the Hindenburg Omen gives a perfect signal there will still be naysayers that will claim it is all luck and voodoo. Right? That's fine, I'm simply trying to help, and I'm sure that there are many out there that appreciate it, even if a few insist on rejecting it.
Post New | Post Reply | Reply Later | Create Poll . Report this Post | Recommend it!
Print the post  
UnThreaded | Threaded | Whole Thread (34) | Ignore Thread Prev | Next

Announcements

Post of the Day:
Macro Economics

2.1: The Labor Pool Problem
What was Your Dumbest Investment?
Share it with us -- and learn from others' stories of flubs.
When Life Gives You Lemons
We all have had hardships and made poor decisions. The important thing is how we respond and grow. Read the story of a Fool who started from nothing, and looks to gain everything.
Community Home
Speak Your Mind, Start Your Blog, Rate Your Stocks

Community Team Fools - who are those TMF's?
Contact Us
Contact Customer Service and other Fool departments here.
Work for Fools?
Winner of the Washingtonian great places to work, and "#1 Media Company to Work For" (BusinessInsider 2011)! Have access to all of TMF's online and email products for FREE, and be paid for your contributions to TMF! Click the link and start your Fool career.
Advertisement