Message Font: Serif | Sans-Serif
No. of Recommendations: 32
There is either an average underperformance or there is not. No one seems to be able to agree whether this is the case, and yet you seem to want us to believe you have all the facts and thus yours is the only valid opinion in the matter.

No you're looking at this from the wrong perspective. Suppose the population (long run) difference in returns between index funds and actively managed funds is X% per year ***. The population includes all past and all future years. Unfortunately we don't know X (obviously), can't even agree whether it's positive, negative or zero, but decide to estimate it, i.e. letting data tell us. We collect data for one year, 2000, where the average mutual fund outperformed the market by 12 percentage points (actual figure). Here's the conversation that follows:

Snoop: OK, our best guess is that X=-12

Otter: Come on you can't measure the population from one year where managed funds were lucky. We need to consider more years that's sound statistical practice.

Snoop: OK I collected data for 1999 also and funds underperformed by 3%. Now the average X (ignore compounding) is (-12+3)/2=-4.5.

Otter: No, no, no. No! we need more information. The statistical uncertainty on your 4.5 figure is huge.

Snoop: Alright let's include 1998 where funds underperformed by 7%. We now have X=(-12+3+7)/3=-0.67, so you see managed funds still outperformed.

Otter: This is annoying. You know very well that we need to collect as much data as possible. We also can't judge the rule maker portfolio based on a few years right?

Snoop: (albaby actually) Right. Russ Wermers collected 19 years of data from 1976-1994 and actively managed funds returned on average .6% less than index funds.

Otter: There you see. If you compound this difference over 40 years you get 35%. index funds underperformed by 35%.

Snoop: Yo Otter, if the very large 12% difference in favor of managed funds based on one year wasn't representative for the population, why do you think this tiny .6% figure in favor of index funds is based on 19 years?

Otter: OK, statistics. It might be a small figure but way more precise when estimated on 19 years worth of data instead of only one.

Snoop: Luckily statistics can estimate the precision for us also. Based on my longer 1962-2000 sample for all funds I computed the standard deviation of the difference in log returns between the market and managed funds for a 19 year sample to be 1.3%. Compare that to the 0.6% average return difference (.58% in log terms) and with a t-statistic of .58/1.3=0.45 we find that the outperformance is located at the 67th percentile in the distribution of outcomes that should occur if there really is no population difference in returns between index and managed funds. There's not a single person in this world with an understanding of statistics that will tell you that this isn't in accordance with the hypostesis that X=0. To get significant outperformance you typically need the t-statistic to be located at the 95th percentile or higher. For this to happen the sample difference between index and managed fund returns would have needed to be at least 1.7% per year. Also, suppose we wanted to test if X=-.6%. The t-test for this hypothesis is at the 81st percentile and this not inconsistent with the data. In other words, if you can say that index funds outperform managed funds by .6% per year based on Wermers sample, I can say that index funds underperform by .6% in the population based on the same evidence.

Otter: But the difference really was .6%!

Snoop: Yes but that's a sample difference and we're interested in the population. The return difference will be different in different samples and you might easily have gotten -.6%.

Otter: Look Mr. I also flipped a coin 19 times and got tail 48% of the times. Thus I have an unfair coin.

Snoop: No Otter that's just a sample. I bet the population odds are 50%.

Otter: No it really was 48%. I wrote down the results of all 19 tosses, see?

Snoop: No, no, no. No! try tossing another 19 times and you'll see.

Otter: I did and got 52%. You are right the sample mean can differ from the population. Hmmm.

Snoop: Yes and likewise for the average difference in return between index and managed funds.

Otter: So what you are saying is that it's not inconsistent with the data to say that a possible outcome over the 19 years following Wermers period is the reverse, namely that managed funds will outperform index funds by .6%.

Snoop: Yes and everyone will agree with this. So yes, we agree that there was an average sample underperformance, but obviously are only interested in the population for which we haven't learned that there's underperformance.

Which study does one listen to? Which one do you believe? Wermers? Fama's?

What are you talking about? There's no disagreement!

Because an annual 65 bp IS meaningful in the real world, even if it is not statistically so,

Of course it's meaningful. If you and I toss a coin for $1 million and you win it's also meaningful (for both of us). However, the figure isn't high enough to say anything meaningful about the population difference in returns between managed and index funds.

I'm sorry Otter but your comments are not unlike what I've gotten from your colleagues on this matter. They are not unlike the comments I got when criticizing the Foolish Four a couple of years ago where people were laughing all the way to bank that ten percentage point or more average historical outperformance which was meaningful to them in the same way the .6% figure in meaningful to you. Boy did they learn from TMF's research study the difference between sample and population average.

*** Now if X varies over time in the population all bets are off. The historical evidence including the .6% average figure isn't informative at all unless we know how it varies.

Print the post  


What was Your Dumbest Investment?
Share it with us -- and learn from others' stories of flubs.
When Life Gives You Lemons
We all have had hardships and made poor decisions. The important thing is how we respond and grow. Read the story of a Fool who started from nothing, and looks to gain everything.
Contact Us
Contact Customer Service and other Fool departments here.
Work for Fools?
Winner of the Washingtonian great places to work, and Glassdoor #1 Company to Work For 2015! Have access to all of TMF's online and email products for FREE, and be paid for your contributions to TMF! Click the link and start your Fool career.