Recommendations: 23
Thanks to several users on this board such as Eldrehad, LoneIguana, and HelicalZz, for helping us devise a new accuracy ranking.
Here's what we plan to implement at public launch:
Players are ranked in terms of accuracy by looking at comparing the probabilities of achieving their respective accuracy ratings.
Before getting into the tricky math, here's the problem we've solved.
In the current BETA version, players are ranked for accuracy solely on the percentage right they have. So a player with 7 out of 7 correct will be ranked ahead of someone who has 99 out of 100. This seems absurd.
So, now, we look at the probabilities of getting X correct and compare those numbers. And the probability of getting 7 out of 7 is about 1% (assuming around a 50% success chance), while the probability of getting 99 out of 100 right is virtually 0%. So, if a player managed to actually make 99 out of 100 correct, he would pretty much be guaranteed to be the most accurate player in CAPS.
99 out of 100 is far too extreme and would likely never happen. More realistically, in our new method, a player who has picked 70 correct out of 100 will still beat a player who has 7 of 7, for example.
That's the gist of it, and now the geeky math stuff follows:
Let's compare players who have 7 picks versus players who have 20 picks.
First, we look at the probability of getting X correct out of 7, which uses the binomial formula (P = probability of) :
P(X correct) = P(correct pick)^(# Correct Picks) * P incorrect pick)^(#Incorrect Picks) * Number of Ways to get X of 7 picks.
This yields the following:
Picks Correct Prob of X correct 7 0 0.781% 7 1 5.469% 7 2 16.406% 7 3 27.344% 7 4 27.344% 7 5 16.406% 7 6 5.469% 7 7 0.781%
Now because it's better to get more correct, we need to make this cumulative.
Picks Correct Prob of X or MORE correct 7 0 100.00% 7 1 99.22% 7 2 93.75% 7 3 77.34% 7 4 50.00% 7 5 22.66% 7 6 6.25% 7 7 0.781%
So, the probability of getting 5 or more correct is sum of all the probabilities with more correct, or P(5 or more) = P(5)+P(6)+P(7)
This assumes that the probability of making a correct pick is 50%. We y may bump this number a little higher or even base it on the actual average accuracy % of CAPS players, which as of last week was around 57%.
Now, let's look at the table for someone with 20 picks:
Picks Correct Prob of X or MORE correct 20 0 100.0000000% 20 1 99.9999046% 20 2 99.9979973% 20 3 99.9798775% 20 4 99.8711586% 20 5 99.4091034% 20 6 97.9305267% 20 7 94.2340851% 20 8 86.8412018% 20 9 74.8277664% 20 10 58.8098526% 20 11 41.1901474% 20 12 25.1722336% 20 13 13.1587982% 20 14 5.7659149% 20 15 2.0694733% 20 16 0.5908966% 20 17 0.1288414% 20 18 0.0201225% 20 19 0.0020027% 20 20 0.0000954%
So, look at our player who has 16 of 20 correct. The likelihood of achieving 16 or more correct is .59%, which is less likely than achieving 7 of 7 (probability of .78%), so the 16 of 20 player would be ranked higher in terms of accuracy.
With this change implemented on our test system, we see players with lots of picks and very high scores like TMF BreakerCharly and TMFEldrehad (who have compiled accuracies of hovering atound 69%) jump back up to our top spots.
We are happy with this (not because they are TMFers... grin), but because it brings the players that are most frequently adding to our community intelligence higher up in the rankings. However, the players with few picks can still do well, but they have a much lower margin for error.
Comments are welcome.
Todd



Announcements
