They said
it couldn't be written down...
They swore
they'd never reveal their secret...
Yet today, ladies and gentlemen, we unveil
Goko's Rating system in a formula!... well, in a few lines of pseudocode anyway:
- Display rating as floor(μ-2σ)
- Update μ and σ using TrueSkill with β=1375, γ=27.5, draw_probability=5%
Yup, after all the fuss and all the secrecy, they've just been running TrueSkill. Unmodified, official, read-the-article, download-the-source-code Microsoft TrueSkill
TM.
Why Goko misled us about this is totally beyond me. I contacted Jeff last week, just in case there was some good reason for the subterfuge and MakingFun would somehow be furious if people knew the actual system. I never got a response... so I guess I hope they're not.
TweaksSo what about all those weird tweaks? Goko told us that they were
"perverting the rating system" to prevent ratings from dropping after a win, and that ratings were
bounded above and below. And what about that
massive overnight
drop in rating?
None of these turn out to be anywhere near as bad as many of us thought. Goko Pro isn't a good system, but it's bad because of its choice of TrueSkill parameters, not because of its tweaks.
- Rating can't go below 0: This tweak is purely cosmetic. If μ-2σ<0, your rating will be displayed as 0, but Goko continues tracking your μ and σ correctly.
- Rating never drops after a win: It's legitimately possible for your displayed TrueSkill rating to go down after a win. If you beat a far lower-rated player, TS gives you a modest increase in σ but only a tiny increase in μ. So μ-2σ can go down even though μ itself (its best guess for your skill) has gone up.
Goko's tweak for this is also purely cosmetic. If your displayed rating would change by -3 after a win, the client just lies to you and says there was no change. But it keeps tracking your μ and σ correctly. Then if you win the next game and deserve +20, it compensates by showing you +17 instead.
- μ can't go below 0 or above 10k: This one's real, at least for the Casual system. But μ=0 in Pro mode is so horrifically bad that not one player has bumped up against the limit. Same story for the alleged upper bound at 10k.
- Daily increase in uncertainty: This used to be really brutal.... maybe a 5% increase in σ every 12:00 AM EST. And the increases compounded since they were re-multiplying your σ by 1.05 every day. So if you were a frequent player and you took a break, you might drop by -20 points on the first day, but then by -21 on the next day, and by the end of a month you'd be dropping nearly 100 points a day. As of now, σ increases by only 1% per day, which is more like -4 points on the first day for a frequent player. Also, it now happens at 12:30 AM... presumably for strategic sheep purposes.
Try it yourself!I coded up their TrueSkill parameters and tweaks so that you can verify it. Just punch in your username and your opponent's, and it'll tell you what to expect after a win, loss, or draw. It tells you both your "real" new rating and the change that MF/Goko will tell you. They can diverge if you beat a much lower rated opponent, as in the example below.
Please post a screen shot if you discover a case in which its prediction is wrong, but first double-check that it actually had your rating right before the game. The process that collects that data can fall behind.
Pro Mode Rating Predictor (Offline)
Continued in
Part 2: Reverse Engineering the System