Dominion Strategy Forum

Please login or register.

Login with username, password and session length
Pages: [1] 2 3 4  All

Author Topic: Discussion on non-random game selection and TrueSkill ranking  (Read 27741 times)

0 Members and 1 Guest are viewing this topic.

jsh357

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2577
  • Shuffle iT Username: jsh357
  • Respect: +4340
    • View Profile
    • JSH Gaming: Original games
0

I checked the leaderboard today and noticed Obi Wan has achieved a perfect 50.  Whoah

Administrator note: this thread was forked from Obi Wan Bonogi's introduction thread
« Last Edit: April 24, 2012, 11:16:21 am by theory »
Logged
Join the Dominion community Discord channel! Chat in text and voice; enter dumb tournaments; spy on top players!

https://discord.gg/2rDpJ4N

WanderingWinder

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5275
  • ...doesn't really matter to me
  • Respect: +4386
    • View Profile
    • WanderingWinder YouTube Page
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #1 on: April 23, 2012, 12:35:34 pm »
0

I checked the leaderboard today and noticed Obi Wan has achieved a perfect 50.  Whoah
Eh, that's just a 50. It's certainly possible to do better, and I'm sure Bonogi would tell you that he isn't playing perfectly. Probably realistic top end is around 53-55, I would guess. Perfect would be nearly 60, if you played enough.

jsh357

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2577
  • Shuffle iT Username: jsh357
  • Respect: +4340
    • View Profile
    • JSH Gaming: Original games
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #2 on: April 23, 2012, 12:37:18 pm »
0

I checked the leaderboard today and noticed Obi Wan has achieved a perfect 50.  Whoah
Eh, that's just a 50. It's certainly possible to do better, and I'm sure Bonogi would tell you that he isn't playing perfectly. Probably realistic top end is around 53-55, I would guess. Perfect would be nearly 60, if you played enough.

I thought that it capped at 50 for some reason... guess not
Logged
Join the Dominion community Discord channel! Chat in text and voice; enter dumb tournaments; spy on top players!

https://discord.gg/2rDpJ4N

greatexpectations

  • Saboteur
  • *****
  • Offline Offline
  • Posts: 1097
  • Respect: +1067
    • View Profile
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #3 on: April 23, 2012, 12:57:50 pm »
0

as i understand it, because of the way TrueSkill works, the peak level should slowly drift upwards over time.  this is a result of more and more players being ranked on the leaderboard. compare:

April 22, 2012 - 8518 ranked players, 44 players ranked lvl 40+
April 22, 2011 - 3764 ranked players, 13 players ranked lvl 40+ (one of whom was likely illegitimate)

Logged
momomoto: ...I looked at the tableau and went "Mountebank? That's for jerks."
rrenaud: Jerks win.

Geronimoo

  • Saboteur
  • *****
  • Offline Offline
  • Posts: 1059
  • Respect: +868
    • View Profile
    • Geronimoo's Dominion Simulator
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #4 on: April 23, 2012, 02:24:47 pm »
0

I checked the leaderboard today and noticed Obi Wan has achieved a perfect 50.  Whoah
Eh, that's just a 50. It's certainly possible to do better, and I'm sure Bonogi would tell you that he isn't playing perfectly. Probably realistic top end is around 53-55, I would guess. Perfect would be nearly 60, if you played enough.
He's biasing Prosperity, maybe if he declines every Big Money game he'll get even higher...
Logged

WanderingWinder

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5275
  • ...doesn't really matter to me
  • Respect: +4386
    • View Profile
    • WanderingWinder YouTube Page
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #5 on: April 23, 2012, 02:30:23 pm »
0

I'm assuming no gaming of the system. Whether what Bonogi does counts as gaming the system or not... well, we can have this debate I guess. But I mean, I could probably be at level 70 tomorrow if I wanted to...

Fabian

  • 2012 Swedish Champion
  • *
  • Offline Offline
  • Posts: 666
  • Respect: +542
    • View Profile
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #6 on: April 23, 2012, 02:45:09 pm »
0

Don't know if "gaming the system" is a useful phrasing, but it seems very clear to me that Obi Wan's results can't be compared to everyone else's, much like (like Geronimoo says) someone who declines every non-engine board, or someone who plays nothing but a pre-determined kingdom every game, or someone who creates dummy accounts which concede to their main account, etc. These are all different games as far as ranking/results go.
Logged

jonts26

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2746
  • Shuffle iT Username: jonts
  • Respect: +3671
    • View Profile
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #7 on: April 23, 2012, 05:12:06 pm »
0

Obi Wan, not biasing prosperity, would sit a bit lower on the leader board, but maybe not that much. 3-5 levels? Unlike previous people who've gamed the leaderboard, Obi Wan is still really really good at dominion.
Logged

O

  • Jester
  • *****
  • Offline Offline
  • Posts: 836
  • Respect: +605
    • View Profile
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #8 on: April 23, 2012, 05:22:18 pm »
0

I'm assuming no gaming of the system. Whether what Bonogi does counts as gaming the system or not... well, we can have this debate I guess. But I mean, I could probably be at level 70 tomorrow if I wanted to...

You'd need a bot army of dummy accounts to feed eachother... Doesn't the system weigh the effect games have partially by skill differential, so games with someone 25 below you are near meaningless?
Logged

Axxle

  • Torturer
  • *****
  • Offline Offline
  • Posts: 1664
  • Most Valuable Serial Killer
  • Respect: +1966
    • View Profile
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #9 on: April 23, 2012, 05:44:35 pm »
+2

Don't know if "gaming the system" is a useful phrasing, but it seems very clear to me that Obi Wan's results can't be compared to everyone else's, much like (like Geronimoo says) someone who declines every non-engine board, or someone who plays nothing but a pre-determined kingdom every game, or someone who creates dummy accounts which concede to their main account, etc. These are all different games as far as ranking/results go.
I have to say this is a terrible topic to be discussing in Obi Wan's introduction thread.
Logged
We might be from all over the world, but "we all talk this one language  : +1 card + 1 action +1 buy , gain , discard, trash... " - RTT

Fabian

  • 2012 Swedish Champion
  • *
  • Offline Offline
  • Posts: 666
  • Respect: +542
    • View Profile
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #10 on: April 23, 2012, 06:03:40 pm »
0

Definitely not questioning that jonts, I've said on a few occasions that the only players I feel play better than I do are you and him*. Whether that's true or not, clearly he's super good.

* Which is not to say that others aren't better too, just that I don't feel outmatched by anyone else.
« Last Edit: April 23, 2012, 06:07:49 pm by Fabian »
Logged

WanderingWinder

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5275
  • ...doesn't really matter to me
  • Respect: +4386
    • View Profile
    • WanderingWinder YouTube Page
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #11 on: April 23, 2012, 09:26:05 pm »
0

I'm assuming no gaming of the system. Whether what Bonogi does counts as gaming the system or not... well, we can have this debate I guess. But I mean, I could probably be at level 70 tomorrow if I wanted to...

You'd need a bot army of dummy accounts to feed eachother... Doesn't the system weigh the effect games have partially by skill differential, so games with someone 25 below you are near meaningless?

I stand by my guesstimate. (evil grin)

olneyce

  • 2011 DS Champion
  • *
  • Offline Offline
  • Posts: 245
  • Respect: +210
    • View Profile
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #12 on: April 24, 2012, 12:30:38 am »
0

I mean, I've got two accounts well beyond level 40.  I could use olneyce to win 85 straight games against The 9th Doctor and it would be worth far more than Karumah's silly games. 

And it only takes a week or two to get a reasonably solid level 40 account if that's your real skill level.  Set up one or two more of them and then beat each of them into submission.  I'd have to think that would put the main account up in the 70s pretty quickly.

It would obviously be totally pointless to actually DO this.  But I don't really think it would be that difficult to manage
Logged

TheMathProf

  • Scout
  • ****
  • Offline Offline
  • Posts: 43
  • Respect: +7
    • View Profile
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #13 on: April 24, 2012, 08:31:12 am »
0

as i understand it, because of the way TrueSkill works, the peak level should slowly drift upwards over time.  this is a result of more and more players being ranked on the leaderboard. compare:

April 22, 2012 - 8518 ranked players, 44 players ranked lvl 40+
April 22, 2011 - 3764 ranked players, 13 players ranked lvl 40+ (one of whom was likely illegitimate)

This same effect as happened with Elo ratings in chess; when Fischer was world champ it was unheard of to have a player rated 2800; currently, there are three such players with Anand knocking at the door at 2799.  As well, the average rating of the top 100 has drifted up 56 points in the last 12 years (http://ratings.fide.com/toplist.phtml?list=men).
Logged

greatexpectations

  • Saboteur
  • *****
  • Offline Offline
  • Posts: 1097
  • Respect: +1067
    • View Profile
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #14 on: April 24, 2012, 09:00:24 am »
0

This same effect as happened with Elo ratings in chess; when Fischer was world champ it was unheard of to have a player rated 2800; currently, there are three such players with Anand knocking at the door at 2799.  As well, the average rating of the top 100 has drifted up 56 points in the last 12 years (http://ratings.fide.com/toplist.phtml?list=men).

TrueSkill was based off of Elo so it isn't really surprising that they function similarly. on here both the peak level and average of the top 100 have drifted upwards as well. the time scale is a lot smaller, but online dominion has probably seen faster growth than competitive chess does.   

I mean, I've got two accounts well beyond level 40.  I could use olneyce to win 85 straight games against The 9th Doctor and it would be worth far more than Karumah's silly games. 

And it only takes a week or two to get a reasonably solid level 40 account if that's your real skill level.  Set up one or two more of them and then beat each of them into submission.  I'd have to think that would put the main account up in the 70s pretty quickly.

at this point i would gladly bet that far more of the top players have alternate accounts than don't have them. i regularly assume that any account over lvl 35 with <1000 (maybe more, maybe less) games played is just somebody's alternate.  i mean, come on now, stan marsh is currently a level 40 with just 314 games played.
Logged
momomoto: ...I looked at the tableau and went "Mountebank? That's for jerks."
rrenaud: Jerks win.

paddyodoors

  • Guest
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #15 on: April 24, 2012, 09:38:32 am »
0

Don't know if "gaming the system" is a useful phrasing, but it seems very clear to me that Obi Wan's results can't be compared to everyone else's, much like (like Geronimoo says) someone who declines every non-engine board, or someone who plays nothing but a pre-determined kingdom every game, or someone who creates dummy accounts which concede to their main account, etc. These are all different games as far as ranking/results go.
I have to say this is a terrible topic to be discussing in Obi Wan's introduction thread.

With respect, I'm not sure I agree.  What better place to talk about someone than right in front of them?  I think it's better than having a huge network of PM-discussions amongst online Dominion Illuminati or something like that.  At least this way, OWB can see what is being asserted about him and respond if he wishes.  Both good things, IMHO.

As for me, I think the current system is good, but not perfect.  I am one of the purists: a player looking for a truly "random" game, no bias whatsoever -- each and every Kingdom card having an equal weight in the game selection.  I don't fault others for having their preferences, but I do wish there was an objective way to measure myself against other players in pure, unadulterated, fully random Dominion.  Call me crazy.
Logged

theory

  • Administrator
  • *****
  • Offline Offline
  • Posts: 3603
  • Respect: +6125
    • View Profile
    • Dominion Strategy
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #16 on: April 24, 2012, 09:41:55 am »
+1

I think there's a big gap between biasing towards Prosperity and playing KC-Goons-Masq 50 games in a row.  Consider what each person's accomplishments represent: OWB is the best in the world, playing Dominion with a slight Prosperity bias.  Karumah is the best in the world, so long as you play a board with KC-Goons-Masq on it against people who don't know better.  Larry is the best in the world, so long as his opponents always lose to him.

I just don't think it's meaningful to shoot for platonically ideal Dominion when Dominion is already somewhat luck-driven.
Logged

paddyodoors

  • Guest
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #17 on: April 24, 2012, 09:57:34 am »
+2

I think there's a big gap between biasing towards Prosperity and playing KC-Goons-Masq 50 games in a row.  Consider what each person's accomplishments represent: OWB is the best in the world, playing Dominion with a slight Prosperity bias.  Karumah is the best in the world, so long as you play a board with KC-Goons-Masq on it against people who don't know better.  Larry is the best in the world, so long as his opponents always lose to him.

I just don't think it's meaningful to shoot for platonically ideal Dominion when Dominion is already somewhat luck-driven.

I agreed 100% with everything in the first paragraph.  Maybe even 9000%.

RE: that last line:
-"platonically ideal" is not a bad thing at all!  It seems to have been given a bad rap over the centuries.  Boo.
-Just because Dominion is luck-driven does not somehow make it less "meaningful" to attempt to find an ideal, balance-amongst-the-chaos way to play...  All other things being equal, I'm hoping to play games where all other things can be equal.
Logged

rrenaud

  • Administrator
  • *****
  • Offline Offline
  • Posts: 991
  • Uncivilized Barbarian of Statistics
  • Respect: +1197
    • View Profile
    • CouncilRoom
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #18 on: April 24, 2012, 10:07:19 am »
+15

Logged

rrenaud

  • Administrator
  • *****
  • Offline Offline
  • Posts: 991
  • Uncivilized Barbarian of Statistics
  • Respect: +1197
    • View Profile
    • CouncilRoom
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #19 on: April 24, 2012, 10:09:44 am »
+2

I think it's reasonable to ask that everyone plays games sampled from the same kingdom distribution in order to have a meaningful ranking, especially at the top, where the differences are so small.

OTOH, I think that among all the top players, my record against Obi Wan Bonogi is the best, so long live OWB ;).
Logged

Geronimoo

  • Saboteur
  • *****
  • Offline Offline
  • Posts: 1059
  • Respect: +868
    • View Profile
    • Geronimoo's Dominion Simulator
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #20 on: April 24, 2012, 10:10:28 am »
+4

I'd like the leaderboard to consist only of people playing auto-match without bias. And the board should be invisible when you decide to accept or decline the match.

Maybe we should do a poll.....???
Logged

DStu

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2627
  • Respect: +1490
    • View Profile
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #21 on: April 24, 2012, 10:14:16 am »
0

I'd like the leaderboard to consist only of people playing auto-match without bias. And the board should be invisible when you decide to accept or decline the match.

Maybe we should do a poll.....???

If I remember correctly, we had this discussion before and the result was the veto mode...
Logged

Fabian

  • 2012 Swedish Champion
  • *
  • Offline Offline
  • Posts: 666
  • Respect: +542
    • View Profile
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #22 on: April 24, 2012, 10:28:29 am »
0

I'm with Geronimoo and rrenaud. I don't see a meaningful distinction between banning pre-determined kingdoms but allowing Colony bias as far as a ranking leaderboard goes. Both are very different games compared to random Dominion, just like various methods of outright cheating (Larry, Karumah, etc) mentioned above. I don't really agree with theory that there's a "slight" Prosperity bias either; Obi Wan plays 49% of his games with Colony compared to 16% when playing random cards iirc? That's a very different game indeed, imo, and it doesn't make for reasonable comparisons between different players, which is why it's weird to me to have it included on a leaderboard, much like the Larrys and the Karumahs etc.
Logged

rrenaud

  • Administrator
  • *****
  • Offline Offline
  • Posts: 991
  • Uncivilized Barbarian of Statistics
  • Respect: +1197
    • View Profile
    • CouncilRoom
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #23 on: April 24, 2012, 10:28:50 am »
0

I'd like the leaderboard to consist only of people playing auto-match without bias. And the board should be invisible when you decide to accept or decline the match.

Maybe we should do a poll.....???

If I remember correctly, we had this discussion before and the result was the veto mode...

If you at first you don't succeed, try, try again?
Logged

rrenaud

  • Administrator
  • *****
  • Offline Offline
  • Posts: 991
  • Uncivilized Barbarian of Statistics
  • Respect: +1197
    • View Profile
    • CouncilRoom
Re: Discussion on non-random game selection and TrueSkill ranking
« Reply #24 on: April 24, 2012, 10:36:16 am »
+4

I'm with Geronimoo and rrenaud. I don't see a meaningful distinction between banning pre-determined kingdoms but allowing Colony bias as far as a ranking leaderboard goes. Both are very different games compared to random Dominion, just like various methods of outright cheating (Larry, Karumah, etc) mentioned above. I don't really agree with theory that there's a "slight" Prosperity bias either; Obi Wan plays 49% of his games with Colony compared to 16% when playing random cards iirc? That's a very different game indeed, imo, and it doesn't make for reasonable comparisons between different players, which is why it's weird to me to have it included on a leaderboard, much like the Larrys and the Karumahs etc.

There are technical solutions to the problem.

For example, you, I, someone could run a ranking system where over-sampled games for a given player are corrected for by decreasing their weight.  Since Colony games count for 49% of Obi Wan's games, and they should only be 16%, then each Colony game updates his rating by only 16/49 of the ordinary update.

http://en.wikipedia.org/wiki/Importance_sampling
Logged
Pages: [1] 2 3 4  All
 

Page created in 0.658 seconds with 20 queries.