Message Font: Serif | Sans-Serif
No. of Recommendations: 7
According to the help page, the CAPs system currently determines your accuracy score by the:

"PROBABILITY of achieving that accuracy and not the actual accuracy. For example, a player who makes 99 out of 100 correct predictions will be ranked ahead of a player who makes 7 out of 7 correct because it is much more difficult to be correct 99 times out of 100".

This has led to some crazy results for some players. Looking at their scores, it would seem that they defy the laws of nature. Save some real insight and stock picking genius, they achieve results that, from a probability standpoint, would supposedly happen randomly in one in a billion tries. Are they really this good?

The flaw in the CAPs scoring system is that it treats picking stock results like picking coin flip results, when there is a huge difference between the two. Mainly: coin flips (fair ones) are independent occurences while companies' stock prices are not necessarily independent and may be correlated.

If I choose 30 stocks from the same industry, their results will, to a large extent, correlate with each other relative to the S&P 500. If I choose 30 gold stocks to outperform over the next few weeks, it is overwhelmingly likely that I will end that time with a skewed distribution connected to the price of gold. If I am "right", I might have 25 out of 30 winning picks -- A huge boost to my accuracy, or 25 out of 30 losing picks -- a huge hit. From an expected value perspective, in theory, this might even out, but from a CAPS scoring perspective it leads to a strange result.

Essentially I have taken a 50-50 proposition: how the gold industry will do over the next few weeks, and leveraged it for a huge accuracy gain. CAPS thinks I just picked 25 out of 30 coin flips correctly (a 1 in thousands chance) when my real odds were as good as someone picking much fewer unrelated companies.

To point out a direct application of the Accuracy Fallacy: I am currently in last place (or very close to it) in the CAPS Kick Off contest. I picked 12-13 companies to underperform in the same industry (alternative energy). It turns out I was exactly wrong. Almost every one of the companies went up instead. Of course, It's a volatile industry. I expected I would have a very high chance of winning (much better than 470 to 1 -- the chance a random player making completely random picks would have). I was wrong in the industry I chose (so far) -- But I think it shows the strategy most players should use to maximize expectation in the contest (if you really can predict how 20 completely disparate companies will perform in the next month with high accuracy -- you don't need the MF newsletter subscriptions).

Anyway -- the point is -- CAPS currently treats someone with 70 out of 100 correct stock picks like he just picked 70 out of 100 coin flips.

A player who picks 70 out of 100 stock picks spread out over several industries is much, much better than a player who picks 75 out of 100 stock picks in one or two industries.

Perhaps CAPs should divide companies into industries or sectors and weight the accuracy scores by performance in those industries/sectors.

Just a thought.
Print the post  


When Life Gives You Lemons
We all have had hardships and made poor decisions. The important thing is how we respond and grow. Read the story of a Fool who started from nothing, and looks to gain everything.
Contact Us
Contact Customer Service and other Fool departments here.
Work for Fools?
Winner of the Washingtonian great places to work, and Glassdoor #1 Company to Work For 2015! Have access to all of TMF's online and email products for FREE, and be paid for your contributions to TMF! Click the link and start your Fool career.