Dominion Strategy Forum

Please login or register.

Login with username, password and session length
Pages: 1 2 3 [4] 5 6 ... 13  All

Author Topic: Isotropish Leaderboard (alternative to Goko Pro)  (Read 144067 times)

0 Members and 1 Guest are viewing this topic.

qmech

  • Torturer
  • *****
  • Offline Offline
  • Posts: 1918
  • Shuffle iT Username: qmech
  • What year is it?
  • Respect: +2320
    • View Profile
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #75 on: July 28, 2013, 10:53:42 am »
0

I had thought that it was just added to the end of the day sigma, but it seems it's figured in using the equation sigma_new = sqrt(sigma_old^2 + gamma^2). Actually, that way makes more sense as it provides a diminishing returns on how fast the variance increases.

sigma^2 is the variance. ;)

(I know you know this!)
Logged

jonts26

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2746
  • Shuffle iT Username: jonts
  • Respect: +3668
    • View Profile
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #76 on: July 28, 2013, 11:13:35 am »
0

I had thought that it was just added to the end of the day sigma, but it seems it's figured in using the equation sigma_new = sqrt(sigma_old^2 + gamma^2). Actually, that way makes more sense as it provides a diminishing returns on how fast the variance increases.

sigma^2 is the variance. ;)

(I know you know this!)


Accurate terminology is for chumps.
Logged

ragingduckd

  • Board Moderator
  • *
  • Offline Offline
  • Posts: 1059
  • Respect: +3527
    • View Profile
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #77 on: July 28, 2013, 02:25:35 pm »
+8

Thanks all for your help. I think I've got the iso TS algorithm now, or at least a close and reasonable variant.

Yed covered most of it but just to clarify, at the end of every day, the sigma parameter for each player is adjusted by the gamma parameter. Gamma is the same for every player, every day and is equal to σ0 / 100 or 0.08333...

This makes sense mathematically, it's consistent with other TS descriptions, and it appears to be what dougz's code does.

I had thought that it was just added to the end of the day sigma, but it seems it's figured in using the equation sigma_new = sqrt(sigma_old^2 + gamma^2). Actually, that way makes more sense as it provides a diminishing returns on how fast the variance increases.

This is a plausible algorithm, but I don't see it in dougz's code or in his description, nor in others' TS descriptions or implementations. The term 02 + γ2) does make an appearance in the ordinary TS algorithm (what Dangauthier et al call "Vanilla" TrueSkill), but it's only in the per-game prior for each player at the top of the factor graph. That affects the post-game values of σ, but only after passing through the whole factor graph and the Bayesian update from the game result.

It's possible that dougz was doing this after each day's results as a modification of TS, though again I don't think that's what the description says. In any case, I just don't like it all that much. I suppose I'll include it if dougz says that it's what isotropic was using and/or if it gives a big boost to predictive performance.

-------

I ran sublee's TS implementation on the Goko data with with μ00=25, β=σ0/2, τ=σ0/100, and ε such that the draw probability is 5%. The parameters that generate my current leaderboard are μ0=25, σ00/3, β=σ0/2, τ=σ0/100, and ε such that the draw probability is 1.75%. Using μ00 is a little unusual, and 1.75% is the draw probability in my dataset, but these differences don't seem to matter much for either the ordering or the predictive performance (see below).

Note that what dougz calls τ isn't the "precision" τ:=1/σ2 but the σ-adjusting term we've been talking about. For reasons I cannot fathom, the TS papers and sublee's code use the symbol τ for both of these values. In dougz's code, he sensible renames the σ-adjustment term to γ (as in the equation at the top of this post).

The mean binomial deviance I calculated from the Goko sample was basically the same regardless of which parameters I used, 0.379 and 0.373 respectively. So that's not strong evidence that either set of parameters is superior. I don't know what mbd Glicko or Elo would generate on this data set, but I expect it's worse. I also expect that there's plenty of room for improvement here.

If assume that isotropic was displaying μ +/- 3σ as the skill range and also using that number for the level, then the leaderboard resulting from my might-be-iso parameters looks a fair amount like the iso leaderboard, and it's ordered a lot like my current implementation:

Code: [Select]
                          Stef - Level 57: 67.64 +/- 10.10 ( 893 games)
                   Mic Qsenoch - Level 53: 63.87 +/- 10.19 ( 837 games)
                    LESPEUTERE - Level 51: 62.05 +/- 10.24 ( 960 games)
                     nomnomnom - Level 50: 64.44 +/- 14.42 ( 118 games)
                    Rene Kuroi - Level 49: 59.74 +/- 10.65 ( 274 games)
                     Geronimoo - Level 48: 58.51 +/- 10.18 ( 717 games)
             Andrew Iannaccone - Level 48: 58.10 +/-  9.99 (1154 games)
              Wandering Winder - Level 48: 58.07 +/- 10.06 ( 854 games)
                           jog - Level 47: 58.04 +/- 10.09 ( 862 games)
                Obi Wan Bonogi - Level 46: 57.10 +/- 10.12 ( 587 games)
                        Fabian - Level 45: 56.12 +/- 10.27 ( 366 games)
                      Boodaloo - Level 45: 58.44 +/- 12.94 ( 134 games)
                      Tao Chen - Level 45: 62.69 +/- 17.44 (  86 games)
                    kenyou2859 - Level 45: 55.59 +/- 10.45 ( 394 games)
                         Rabid - Level 44: 54.98 +/- 10.11 ( 508 games)
                  SheCantSayNo - Level 44: 54.81 +/-  9.95 (1781 games)
                     blueblimp - Level 44: 55.34 +/- 10.89 ( 261 games)
              HiveMindEmulator - Level 44: 56.90 +/- 12.53 ( 149 games)
                       eliegel - Level 44: 54.33 +/- 10.30 ( 398 games)
                Stealth Tomato - Level 43: 54.07 +/- 10.24 ( 573 games)
                    shark_bait - Level 43: 54.48 +/- 10.78 ( 243 games)
                        Jeebus - Level 42: 53.45 +/- 10.54 ( 300 games)
                         manzi - Level 42: 53.03 +/- 10.18 ( 356 games)
              Mike Harris.0001 - Level 42: 53.08 +/- 10.24 ( 423 games)

The major remaining difference is that the 3σ values are higher and more tightly clustered than on the iso board, but that's consistent with the smaller number and variation in number of games played.

Summary:

I find it pretty plausible that isotropic was using "Vanilla" TrueSkill with the parameters above. The difference between those parameters and the ones I would choose doesn't seem large, and I see no real justification for any major deviation from the standard algorithm. Until I hear otherwise from dougz or unless someone else has a compelling argument to the contrary, I'm going to change my leaderboard to use those parameters.
Logged
Salvager Extension | Isotropish Leaderboard | Game Data | Log Search & other toys | Salvager Bug Reports

Salvager not working for me at all today. ... Please help! I can't go back to playing without it like an animal!

shark_bait

  • Saboteur
  • *****
  • Offline Offline
  • Posts: 1103
  • Shuffle iT Username: shark_bait
  • Luckyfin and Land of Hinter for iso aliases
  • Respect: +1868
    • View Profile
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #78 on: July 28, 2013, 02:51:28 pm »
0

Ha ha, this leaderboard is so schizophrenic right now, I'm guessing it's catching up and parsing data?
Logged
Hello.  Name's Bruce.  It's all right.  I understand.  Why trust a shark, right?

Is quite curious - Who is the mystical "Celestial Chameleon"?

ragingduckd

  • Board Moderator
  • *
  • Offline Offline
  • Posts: 1059
  • Respect: +3527
    • View Profile
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #79 on: July 28, 2013, 03:04:59 pm »
+3

Ha ha, this leaderboard is so schizophrenic right now, I'm guessing it's catching up and parsing data?

Yes, it can't make up it's mind. :D

Edit: All caught up. Now using TS with the parameters described two posts up, also implemented sortable rows, # of games, and those gray "Level" entries.
« Last Edit: July 28, 2013, 06:35:36 pm by ragingduckd »
Logged
Salvager Extension | Isotropish Leaderboard | Game Data | Log Search & other toys | Salvager Bug Reports

Salvager not working for me at all today. ... Please help! I can't go back to playing without it like an animal!

sudgy

  • Cartographer
  • *****
  • Offline Offline
  • Posts: 3431
  • Shuffle iT Username: sudgy
  • It's pronounced "SOO-jee"
  • Respect: +2706
    • View Profile
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #80 on: July 28, 2013, 03:19:58 pm »
0

Aw, now I got demoted even more than I expected. :(

Oh well, now my main question is:  "DID STEF GO SUPERHUMAN?!?"
Logged
If you're wondering what my avatar is, watch this.

Check out my logic puzzle blog!

   Quote from: sudgy on June 31, 2011, 11:47:46 pm

Jdaki

  • Bishop
  • ****
  • Offline Offline
  • Posts: 118
  • Respect: +47
    • View Profile
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #81 on: July 28, 2013, 03:24:37 pm »
0

Although I've wasted the last who knows how long minutes reading all the gammas and alphas and mus without a clue, this leader-board is brilliant work. Kudos to Andrew.
Logged

Awaclus

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 11809
  • Shuffle iT Username: Awaclus
  • (´。• ω •。`)
  • Respect: +12846
    • View Profile
    • Birds of Necama
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #82 on: July 28, 2013, 03:58:37 pm »
+2

Aw, now I got demoted even more than I expected. :(

Oh well, now my main question is:  "DID STEF GO SUPERHUMAN?!?"
Stef has always been superhuman. My main question is: "DID STEF GO SUPERSTEF?!?"
Logged
Bomb, Cannon, and many of the Gunpowder cards can strongly effect gameplay, particularly in a destructive way

The YouTube channel where I make musicDownload my band's Creative Commons albums for free

WanderingWinder

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5275
  • ...doesn't really matter to me
  • Respect: +4381
    • View Profile
    • WanderingWinder YouTube Page
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #83 on: July 28, 2013, 04:06:05 pm »
0

Is anyone else having the problem that this site is almost without fail causing crashes/extraordinarily high lag?

Awaclus

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 11809
  • Shuffle iT Username: Awaclus
  • (´。• ω •。`)
  • Respect: +12846
    • View Profile
    • Birds of Necama
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #84 on: July 28, 2013, 04:08:15 pm »
0

Is anyone else having the problem that this site is almost without fail causing crashes/extraordinarily high lag?
I don't know if anyone else is having the problem, but it has been working quite well for me.
Logged
Bomb, Cannon, and many of the Gunpowder cards can strongly effect gameplay, particularly in a destructive way

The YouTube channel where I make musicDownload my band's Creative Commons albums for free

ragingduckd

  • Board Moderator
  • *
  • Offline Offline
  • Posts: 1059
  • Respect: +3527
    • View Profile
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #85 on: July 28, 2013, 04:21:51 pm »
0

Is anyone else having the problem that this site is almost without fail causing crashes/extraordinarily high lag?
I don't know if anyone else is having the problem, but it has been working quite well for me.

The table-sorting plugin is a bit of a memory hog... probably isn't really designed to deal with 8k rows. I guess it might be a problem with mobile devices, or maybe if you're running a JS debugger or certain browser add-ons.

In any case, I just disabled the sortable columns. See if it's still a problem.
Logged
Salvager Extension | Isotropish Leaderboard | Game Data | Log Search & other toys | Salvager Bug Reports

Salvager not working for me at all today. ... Please help! I can't go back to playing without it like an animal!

WanderingWinder

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5275
  • ...doesn't really matter to me
  • Respect: +4381
    • View Profile
    • WanderingWinder YouTube Page
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #86 on: July 28, 2013, 04:30:40 pm »
0

Is anyone else having the problem that this site is almost without fail causing crashes/extraordinarily high lag?
I don't know if anyone else is having the problem, but it has been working quite well for me.

The table-sorting plugin is a bit of a memory hog... probably isn't really designed to deal with 8k rows. I guess it might be a problem with mobile devices, or maybe if you're running a JS debugger or certain browser add-ons.

In any case, I just disabled the sortable columns. See if it's still a problem.
It's been a problem almost every time I refreshed it since before the sorter, though it was usually just that it would be 15ish seconds of I can't do anything. The sorting made it WAY worse and totally froze me out to where I had to stop the browser's process. Just tried it again and first time it took 10 seconds or so, second time it was loading for a minute and I gave up on it. FWIW, it doesn't seem to be a memory thing - it's making my processor usage go up to the max I am letting my browser use (25% of my total processing power). And I am on a laptop.


Edit: Okay, seems to not really be an issue if I bring the page up in chrome (well, the sorting thing would still squawk and take forever to actually sort, but it wasn't locking me up). But the problems I am experiencing are in firefox (22.0), which is weird because iirc that's what you use to test....
« Last Edit: July 28, 2013, 04:59:04 pm by WanderingWinder »
Logged

ragingduckd

  • Board Moderator
  • *
  • Offline Offline
  • Posts: 1059
  • Respect: +3527
    • View Profile
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #87 on: July 28, 2013, 05:08:33 pm »
0

Is anyone else having the problem that this site is almost without fail causing crashes/extraordinarily high lag?
I don't know if anyone else is having the problem, but it has been working quite well for me.

The table-sorting plugin is a bit of a memory hog... probably isn't really designed to deal with 8k rows. I guess it might be a problem with mobile devices, or maybe if you're running a JS debugger or certain browser add-ons.

In any case, I just disabled the sortable columns. See if it's still a problem.
It's been a problem almost every time I refreshed it since before the sorter, though it was usually just that it would be 15ish seconds of I can't do anything. The sorting made it WAY worse and totally froze me out to where I had to stop the browser's process. Just tried it again and first time it took 10 seconds or so, second time it was loading for a minute and I gave up on it. FWIW, it doesn't seem to be a memory thing - it's making my processor usage go up to the max I am letting my browser use (25% of my total processing power). And I am on a laptop.


Edit: Okay, seems to not really be an issue if I bring the page up in chrome (well, the sorting thing would still squawk and take forever to actually sort, but it wasn't locking me up). But the problems I am experiencing are in firefox (22.0), which is weird because iirc that's what you use to test....

Yes, I have no problems in FF. There's no client-side code anymore, so it shouldn't be taking any CPU. It's an unusually large page though (~2MB), which isn't a problem in itself, but I suppose it's possible that some FF Add-on is causing trouble, if it processes pages in a particularly CPU-intensive way. I have an appreciable delay when opening up Firebug, for example. Even if that's the problem though, the real solution would be for me to paginate the leaderboard so it's not such a massive page.
Logged
Salvager Extension | Isotropish Leaderboard | Game Data | Log Search & other toys | Salvager Bug Reports

Salvager not working for me at all today. ... Please help! I can't go back to playing without it like an animal!

soulnet

  • Mountebank
  • *****
  • Offline Offline
  • Posts: 2142
  • Respect: +1751
    • View Profile
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #88 on: July 28, 2013, 05:33:56 pm »
0

Yes, I have no problems in FF. There's no client-side code anymore, so it shouldn't be taking any CPU. It's an unusually large page though (~2MB), which isn't a problem in itself, but I suppose it's possible that some FF Add-on is causing trouble, if it processes pages in a particularly CPU-intensive way. I have an appreciable delay when opening up Firebug, for example. Even if that's the problem though, the real solution would be for me to paginate the leaderboard so it's not such a massive page.

Unless is causing trouble to the server, please don't do that, or at least keep the option to get the entire table. Being able to ctrl+f through it and browse up and down to look who you know or just played is close is great.
Logged

HiveMindEmulator

  • Mountebank
  • *****
  • Offline Offline
  • Posts: 2222
  • Respect: +2118
    • View Profile
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #89 on: July 28, 2013, 05:47:38 pm »
+1

Maybe a dumb question, but why is the number of games so different from the number of games listed in the goko stats (when you click on avatars in the lobby)? Does the latter include casual or adventures or are some pro games excluded from these rankings?
Logged

Kirian

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 7096
  • Shuffle iT Username: Kirian
  • An Unbalanced Equation
  • Respect: +9411
    • View Profile
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #90 on: July 28, 2013, 05:56:11 pm »
0

Maybe a dumb question, but why is the number of games so different from the number of games listed in the goko stats (when you click on avatars in the lobby)? Does the latter include casual or adventures or are some pro games excluded from these rankings?

The latter includes casual and adventures, plus games against bots.
Logged
Kirian's Law of f.DS jokes:  Any sufficiently unexplained joke is indistinguishable from serious conversation.

markusin

  • Cartographer
  • *****
  • Offline Offline
  • Posts: 3846
  • Shuffle iT Username: markusin
  • I also switched from Starcraft
  • Respect: +2437
    • View Profile
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #91 on: July 28, 2013, 06:08:00 pm »
0

Maybe a dumb question, but why is the number of games so different from the number of games listed in the goko stats (when you click on avatars in the lobby)? Does the latter include casual or adventures or are some pro games excluded from these rankings?

The latter includes casual and adventures, plus games against bots.
Yes it even includes adventure games and bot games. At first I though that was kind of silly, but then I figured that every game adds to a player's experience, whether the opponent was a bot or not. It would be more deceptive to see someone who played 100+ bot games appeared as someone with less than 10 games played on their profile.
Logged

ragingduckd

  • Board Moderator
  • *
  • Offline Offline
  • Posts: 1059
  • Respect: +3527
    • View Profile
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #92 on: July 28, 2013, 06:12:40 pm »
+1

Yes, I have no problems in FF. There's no client-side code anymore, so it shouldn't be taking any CPU. It's an unusually large page though (~2MB), which isn't a problem in itself, but I suppose it's possible that some FF Add-on is causing trouble, if it processes pages in a particularly CPU-intensive way. I have an appreciable delay when opening up Firebug, for example. Even if that's the problem though, the real solution would be for me to paginate the leaderboard so it's not such a massive page.

Unless is causing trouble to the server, please don't do that, or at least keep the option to get the entire table. Being able to ctrl+f through it and browse up and down to look who you know or just played is close is great

I won't paginate unless I can find a graceful solution. Sorting a 8k row table in JS isn't very graceful though. I'll keep the option to display the whole table regardless.  No server load problems so far.
Logged
Salvager Extension | Isotropish Leaderboard | Game Data | Log Search & other toys | Salvager Bug Reports

Salvager not working for me at all today. ... Please help! I can't go back to playing without it like an animal!

HiveMindEmulator

  • Mountebank
  • *****
  • Offline Offline
  • Posts: 2222
  • Respect: +2118
    • View Profile
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #93 on: July 28, 2013, 06:32:21 pm »
0

Maybe a dumb question, but why is the number of games so different from the number of games listed in the goko stats (when you click on avatars in the lobby)? Does the latter include casual or adventures or are some pro games excluded from these rankings?

The latter includes casual and adventures, plus games against bots.
Yes it even includes adventure games and bot games. At first I though that was kind of silly, but then I figured that every game adds to a player's experience, whether the opponent was a bot or not. It would be more deceptive to see someone who played 100+ bot games appeared as someone with less than 10 games played on their profile.

Adventure games is a little weird though, since they're not really like real games. But that's not a subject for this thread I guess...
Logged

florrat

  • Minion
  • *****
  • Offline Offline
  • Posts: 542
  • Shuffle iT Username: florrat
  • Respect: +748
    • View Profile
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #94 on: July 28, 2013, 06:33:17 pm »
+1

I won't paginate unless I can find a graceful solution. Sorting a 8k row table in JS isn't very graceful though. I'll keep the option to display the whole table regardless.  No server load problems so far.

Maybe exclude all players with less than 10 (or so) played games? And/or maybe exclude all players who haven't played for over a month?

That should make the table significantly smaller, while keeping almost all interesting data there.
Logged

GeoLib

  • Jester
  • *****
  • Offline Offline
  • Posts: 965
  • Respect: +1265
    • View Profile
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #95 on: July 28, 2013, 10:20:25 pm »
+4

Aw, now I got demoted even more than I expected. :(

Oh well, now my main question is:  "DID STEF GO SUPERHUMAN?!?"
Stef has always been superhuman. My main question is: "DID STEF GO SUPERSTEF?!?"

Logged
"All advice is awful"
 —Count Grishnakh

jonts26

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2746
  • Shuffle iT Username: jonts
  • Respect: +3668
    • View Profile
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #96 on: July 28, 2013, 10:57:42 pm »
0

The variance values in the new leaderboard are interesting. There seems to be a pretty hard floor right around 10/3 which is reached at about 600ish games played. Is there some sort of theoretical limit to how low sigma can get based on the model and is 10/3 a good value for that? My intuition doesn't tell me much here.
Logged

mail-mi

  • Saboteur
  • *****
  • Offline Offline
  • Posts: 1298
  • Shuffle iT Username: mail-mi
  • Come play some Forum Mafia with us!
  • Respect: +1364
    • View Profile
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #97 on: July 28, 2013, 11:32:20 pm »
0

I was about level 1 on iso.

Now I'm 22. I feel so proud.
never mind 10.  :-\
Logged
I currently imagine mail-mi wearing a dark trenchcoat and a bowler hat, hunched over a bit, toothpick in his mouth, holding a gun in his pocket.  One bead of sweat trickling down his nose.

'And what is it that ye shall hope for? Behold I say unto you that ye shall have hope through the atonement of Christ and the power of his resurrection, to be raised unto life eternal, and this because of your faith in him according to the promise." - Moroni 7:41, the Book of Mormon

sudgy

  • Cartographer
  • *****
  • Offline Offline
  • Posts: 3431
  • Shuffle iT Username: sudgy
  • It's pronounced "SOO-jee"
  • Respect: +2706
    • View Profile
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #98 on: July 28, 2013, 11:48:39 pm »
0

I was about level 1 on iso.

Now I'm 22. I feel so proud.
never mind 10.  :-\

Wait, now I'm seven.  I was 23 earlier, and on iso I was 13...
Logged
If you're wondering what my avatar is, watch this.

Check out my logic puzzle blog!

   Quote from: sudgy on June 31, 2011, 11:47:46 pm

eliegel34

  • Golem
  • ****
  • Offline Offline
  • Posts: 175
  • Respect: +146
    • View Profile
Re: Isotropish Leaderboard (alternative to Goko Pro)
« Reply #99 on: July 29, 2013, 02:28:36 am »
+2

Is anyone else having the problem that this site is almost without fail causing crashes/extraordinarily high lag?

Yes, I am having 1/5 games crash right now.

Edit: My 99th post is reply 99 to this thread; neat :)
« Last Edit: July 29, 2013, 02:30:26 am by eliegel34 »
Logged
Pages: 1 2 3 [4] 5 6 ... 13  All
 

Page created in 0.06 seconds with 21 queries.