Dominion Strategy Forum

Dominion => Dominion Online at Shuffle iT => Dominion General Discussion => Dominion Isotropic => Topic started by: jsh357 on April 23, 2012, 12:32:09 pm

Title: Discussion on non-random game selection and TrueSkill ranking
Post by: jsh357 on April 23, 2012, 12:32:09 pm
I checked the leaderboard today and noticed Obi Wan has achieved a perfect 50.  Whoah

Administrator note: this thread was forked from Obi Wan Bonogi's introduction thread (http://forum.dominionstrategy.com/index.php?topic=2254.0)
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: WanderingWinder on April 23, 2012, 12:35:34 pm
I checked the leaderboard today and noticed Obi Wan has achieved a perfect 50.  Whoah
Eh, that's just a 50. It's certainly possible to do better, and I'm sure Bonogi would tell you that he isn't playing perfectly. Probably realistic top end is around 53-55, I would guess. Perfect would be nearly 60, if you played enough.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: jsh357 on April 23, 2012, 12:37:18 pm
I checked the leaderboard today and noticed Obi Wan has achieved a perfect 50.  Whoah
Eh, that's just a 50. It's certainly possible to do better, and I'm sure Bonogi would tell you that he isn't playing perfectly. Probably realistic top end is around 53-55, I would guess. Perfect would be nearly 60, if you played enough.

I thought that it capped at 50 for some reason... guess not
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: greatexpectations on April 23, 2012, 12:57:50 pm
as i understand it, because of the way TrueSkill (http://research.microsoft.com/en-us/projects/trueskill/) works, the peak level should slowly drift upwards over time.  this is a result of more and more players being ranked on the leaderboard. compare:

April 22, 2012 (http://bggdl.square7.ch/leaderboard/leaderboard-2012-04-22.html) - 8518 ranked players, 44 players ranked lvl 40+
April 22, 2011 (http://bggdl.square7.ch/leaderboard/leaderboard-2011-04-22.html) - 3764 ranked players, 13 players ranked lvl 40+ (one of whom was likely illegitimate)

Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: Geronimoo on April 23, 2012, 02:24:47 pm
I checked the leaderboard today and noticed Obi Wan has achieved a perfect 50.  Whoah
Eh, that's just a 50. It's certainly possible to do better, and I'm sure Bonogi would tell you that he isn't playing perfectly. Probably realistic top end is around 53-55, I would guess. Perfect would be nearly 60, if you played enough.
He's biasing Prosperity, maybe if he declines every Big Money game he'll get even higher...
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: WanderingWinder on April 23, 2012, 02:30:23 pm
I'm assuming no gaming of the system. Whether what Bonogi does counts as gaming the system or not... well, we can have this debate I guess. But I mean, I could probably be at level 70 tomorrow if I wanted to...
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: Fabian on April 23, 2012, 02:45:09 pm
Don't know if "gaming the system" is a useful phrasing, but it seems very clear to me that Obi Wan's results can't be compared to everyone else's, much like (like Geronimoo says) someone who declines every non-engine board, or someone who plays nothing but a pre-determined kingdom every game, or someone who creates dummy accounts which concede to their main account, etc. These are all different games as far as ranking/results go.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: jonts26 on April 23, 2012, 05:12:06 pm
Obi Wan, not biasing prosperity, would sit a bit lower on the leader board, but maybe not that much. 3-5 levels? Unlike previous people who've gamed the leaderboard, Obi Wan is still really really good at dominion.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: O on April 23, 2012, 05:22:18 pm
I'm assuming no gaming of the system. Whether what Bonogi does counts as gaming the system or not... well, we can have this debate I guess. But I mean, I could probably be at level 70 tomorrow if I wanted to...

You'd need a bot army of dummy accounts to feed eachother... Doesn't the system weigh the effect games have partially by skill differential, so games with someone 25 below you are near meaningless?
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: Axxle on April 23, 2012, 05:44:35 pm
Don't know if "gaming the system" is a useful phrasing, but it seems very clear to me that Obi Wan's results can't be compared to everyone else's, much like (like Geronimoo says) someone who declines every non-engine board, or someone who plays nothing but a pre-determined kingdom every game, or someone who creates dummy accounts which concede to their main account, etc. These are all different games as far as ranking/results go.
I have to say this is a terrible topic to be discussing in Obi Wan's introduction thread.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: Fabian on April 23, 2012, 06:03:40 pm
Definitely not questioning that jonts, I've said on a few occasions that the only players I feel play better than I do are you and him*. Whether that's true or not, clearly he's super good.

* Which is not to say that others aren't better too, just that I don't feel outmatched by anyone else.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: WanderingWinder on April 23, 2012, 09:26:05 pm
I'm assuming no gaming of the system. Whether what Bonogi does counts as gaming the system or not... well, we can have this debate I guess. But I mean, I could probably be at level 70 tomorrow if I wanted to...

You'd need a bot army of dummy accounts to feed eachother... Doesn't the system weigh the effect games have partially by skill differential, so games with someone 25 below you are near meaningless?

I stand by my guesstimate. (evil grin)
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: olneyce on April 24, 2012, 12:30:38 am
I mean, I've got two accounts well beyond level 40.  I could use olneyce to win 85 straight games against The 9th Doctor and it would be worth far more than Karumah's silly games. 

And it only takes a week or two to get a reasonably solid level 40 account if that's your real skill level.  Set up one or two more of them and then beat each of them into submission.  I'd have to think that would put the main account up in the 70s pretty quickly.

It would obviously be totally pointless to actually DO this.  But I don't really think it would be that difficult to manage
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: TheMathProf on April 24, 2012, 08:31:12 am
as i understand it, because of the way TrueSkill (http://research.microsoft.com/en-us/projects/trueskill/) works, the peak level should slowly drift upwards over time.  this is a result of more and more players being ranked on the leaderboard. compare:

April 22, 2012 (http://bggdl.square7.ch/leaderboard/leaderboard-2012-04-22.html) - 8518 ranked players, 44 players ranked lvl 40+
April 22, 2011 (http://bggdl.square7.ch/leaderboard/leaderboard-2011-04-22.html) - 3764 ranked players, 13 players ranked lvl 40+ (one of whom was likely illegitimate)

This same effect as happened with Elo ratings in chess; when Fischer was world champ it was unheard of to have a player rated 2800; currently, there are three such players with Anand knocking at the door at 2799.  As well, the average rating of the top 100 has drifted up 56 points in the last 12 years (http://ratings.fide.com/toplist.phtml?list=men (http://ratings.fide.com/toplist.phtml?list=men)).
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: greatexpectations on April 24, 2012, 09:00:24 am
This same effect as happened with Elo ratings in chess; when Fischer was world champ it was unheard of to have a player rated 2800; currently, there are three such players with Anand knocking at the door at 2799.  As well, the average rating of the top 100 has drifted up 56 points in the last 12 years (http://ratings.fide.com/toplist.phtml?list=men (http://ratings.fide.com/toplist.phtml?list=men)).

TrueSkill was based off of Elo so it isn't really surprising that they function similarly. on here both the peak level and average of the top 100 have drifted upwards as well. the time scale is a lot smaller, but online dominion has probably seen faster growth than competitive chess does.   

I mean, I've got two accounts well beyond level 40.  I could use olneyce to win 85 straight games against The 9th Doctor and it would be worth far more than Karumah's silly games. 

And it only takes a week or two to get a reasonably solid level 40 account if that's your real skill level.  Set up one or two more of them and then beat each of them into submission.  I'd have to think that would put the main account up in the 70s pretty quickly.

at this point i would gladly bet that far more of the top players have alternate accounts than don't have them. i regularly assume that any account over lvl 35 with <1000 (maybe more, maybe less) games played is just somebody's alternate.  i mean, come on now, stan marsh is currently a level 40 with just 314 games played.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: paddyodoors on April 24, 2012, 09:38:32 am
Don't know if "gaming the system" is a useful phrasing, but it seems very clear to me that Obi Wan's results can't be compared to everyone else's, much like (like Geronimoo says) someone who declines every non-engine board, or someone who plays nothing but a pre-determined kingdom every game, or someone who creates dummy accounts which concede to their main account, etc. These are all different games as far as ranking/results go.
I have to say this is a terrible topic to be discussing in Obi Wan's introduction thread.

With respect, I'm not sure I agree.  What better place to talk about someone than right in front of them?  I think it's better than having a huge network of PM-discussions amongst online Dominion Illuminati or something like that.  At least this way, OWB can see what is being asserted about him and respond if he wishes.  Both good things, IMHO.

As for me, I think the current system is good, but not perfect.  I am one of the purists: a player looking for a truly "random" game, no bias whatsoever -- each and every Kingdom card having an equal weight in the game selection.  I don't fault others for having their preferences, but I do wish there was an objective way to measure myself against other players in pure, unadulterated, fully random Dominion.  Call me crazy.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: theory on April 24, 2012, 09:41:55 am
I think there's a big gap between biasing towards Prosperity and playing KC-Goons-Masq 50 games in a row.  Consider what each person's accomplishments represent: OWB is the best in the world, playing Dominion with a slight Prosperity bias.  Karumah is the best in the world, so long as you play a board with KC-Goons-Masq on it against people who don't know better.  Larry is the best in the world, so long as his opponents always lose to him.

I just don't think it's meaningful to shoot for platonically ideal Dominion when Dominion is already somewhat luck-driven.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: paddyodoors on April 24, 2012, 09:57:34 am
I think there's a big gap between biasing towards Prosperity and playing KC-Goons-Masq 50 games in a row.  Consider what each person's accomplishments represent: OWB is the best in the world, playing Dominion with a slight Prosperity bias.  Karumah is the best in the world, so long as you play a board with KC-Goons-Masq on it against people who don't know better.  Larry is the best in the world, so long as his opponents always lose to him.

I just don't think it's meaningful to shoot for platonically ideal Dominion when Dominion is already somewhat luck-driven.

I agreed 100% with everything in the first paragraph.  Maybe even 9000%.

RE: that last line:
-"platonically ideal" is not a bad thing at all!  It seems to have been given a bad rap over the centuries.  Boo.
-Just because Dominion is luck-driven does not somehow make it less "meaningful" to attempt to find an ideal, balance-amongst-the-chaos way to play...  All other things being equal, I'm hoping to play games where all other things can be equal.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: rrenaud on April 24, 2012, 10:07:19 am
(http://assets.diylol.com/hfs/29b/08a/6a1/resized/beatdown-meme-generator-hi-obi-wan-bonogi-welcome-to-f-ds-fb612c.jpg?1335276366.jpg)
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: rrenaud on April 24, 2012, 10:09:44 am
I think it's reasonable to ask that everyone plays games sampled from the same kingdom distribution in order to have a meaningful ranking, especially at the top, where the differences are so small.

OTOH, I think that among all the top players, my record against Obi Wan Bonogi is the best, so long live OWB ;).
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: Geronimoo on April 24, 2012, 10:10:28 am
I'd like the leaderboard to consist only of people playing auto-match without bias. And the board should be invisible when you decide to accept or decline the match.

Maybe we should do a poll.....???
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: DStu on April 24, 2012, 10:14:16 am
I'd like the leaderboard to consist only of people playing auto-match without bias. And the board should be invisible when you decide to accept or decline the match.

Maybe we should do a poll.....???

If I remember correctly, we had this discussion before and the result was the veto mode...
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: Fabian on April 24, 2012, 10:28:29 am
I'm with Geronimoo and rrenaud. I don't see a meaningful distinction between banning pre-determined kingdoms but allowing Colony bias as far as a ranking leaderboard goes. Both are very different games compared to random Dominion, just like various methods of outright cheating (Larry, Karumah, etc) mentioned above. I don't really agree with theory that there's a "slight" Prosperity bias either; Obi Wan plays 49% of his games with Colony compared to 16% when playing random cards iirc? That's a very different game indeed, imo, and it doesn't make for reasonable comparisons between different players, which is why it's weird to me to have it included on a leaderboard, much like the Larrys and the Karumahs etc.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: rrenaud on April 24, 2012, 10:28:50 am
I'd like the leaderboard to consist only of people playing auto-match without bias. And the board should be invisible when you decide to accept or decline the match.

Maybe we should do a poll.....???

If I remember correctly, we had this discussion before and the result was the veto mode...

If you at first you don't succeed, try, try again?
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: rrenaud on April 24, 2012, 10:36:16 am
I'm with Geronimoo and rrenaud. I don't see a meaningful distinction between banning pre-determined kingdoms but allowing Colony bias as far as a ranking leaderboard goes. Both are very different games compared to random Dominion, just like various methods of outright cheating (Larry, Karumah, etc) mentioned above. I don't really agree with theory that there's a "slight" Prosperity bias either; Obi Wan plays 49% of his games with Colony compared to 16% when playing random cards iirc? That's a very different game indeed, imo, and it doesn't make for reasonable comparisons between different players, which is why it's weird to me to have it included on a leaderboard, much like the Larrys and the Karumahs etc.

There are technical solutions to the problem.

For example, you, I, someone could run a ranking system where over-sampled games for a given player are corrected for by decreasing their weight.  Since Colony games count for 49% of Obi Wan's games, and they should only be 16%, then each Colony game updates his rating by only 16/49 of the ordinary update.

http://en.wikipedia.org/wiki/Importance_sampling
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: paddyodoors on April 24, 2012, 10:38:30 am
Obi Wan Bonogi, I'm 99% certain no one here holds any ill-will towards you... we have been using your example to spur on a larger discussion encompassing things that have nothing to do with you directly.  No insult or offense is intended.

Assuming you're still reading your intro thread, what are your thoughts on all of this?  I'd like to know...

Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: paddyodoors on April 24, 2012, 10:39:24 am
There are technical solutions to the problem.

For example, you, I, someone could run a ranking system where over-sampled games for a given player are corrected for by decreasing their weight.  Since Colony games count for 49% of Obi Wan's games, and they should only be 16%, then each Colony game updates his rating by only 16/49 of the ordinary update.

http://en.wikipedia.org/wiki/Importance_sampling

That suggestion is so... uncivilized. 8)
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: Galzria on April 24, 2012, 10:40:30 am
When I look for games personally, the only things I change are "ranked players only", and "point counter prohibited". I think veto mode is as much a cheat as selecting bias, or as declining games until you get a board you like.

The most unbiased model, I think, would be only to count random (no bias) games (not proposed between players, who could game the system) between two opponents whose name and rank are, along with the board, completely hidden until both sides accept. A player resigning would lose the game, while the remaining player earns nothing (yes, it allows win stealing, but takes away unearned win potential). The games only count for wins when ended on piles or province/colony.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: greatexpectations on April 24, 2012, 10:41:57 am
I'm with Geronimoo and rrenaud. I don't see a meaningful distinction between banning pre-determined kingdoms but allowing Colony bias as far as a ranking leaderboard goes. Both are very different games compared to random Dominion, just like various methods of outright cheating (Larry, Karumah, etc) mentioned above. I don't really agree with theory that there's a "slight" Prosperity bias either; Obi Wan plays 49% of his games with Colony compared to 16% when playing random cards iirc? That's a very different game indeed, imo, and it doesn't make for reasonable comparisons between different players, which is why it's weird to me to have it included on a leaderboard, much like the Larrys and the Karumahs etc.

when i ran the numbers (http://forum.dominionstrategy.com/index.php?topic=2242.msg35209#msg35209) a week ago it was 58.7% colony games vs a 19.8% average.  that is a hair shy of 3x the public average.  he maintains a 1.32 win rate on colony boards as compared to a 1.27 rate on all boards.

as jonts26 and Fabian pointed out though, the difference between OWB and karumah and larry and all those guys is that OWB is a really good player. part of me is actually proud/jealous of the guy for figuring out a clean way to give himself a bit of a competitive edge. 

i think we would all like a way of playing binding fully random games without seeing the set, but until then we will have these issues.  realistically, veto mode is no better (arguably worse) than deliberately choosing a strategic bias. smart players will know how to take advantage of veto mode to give themselves the best chance of winning.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: DStu on April 24, 2012, 10:48:42 am
I'm with Geronimoo and rrenaud. I don't see a meaningful distinction between banning pre-determined kingdoms but allowing Colony bias as far as a ranking leaderboard goes. Both are very different games compared to random Dominion, just like various methods of outright cheating (Larry, Karumah, etc) mentioned above. I don't really agree with theory that there's a "slight" Prosperity bias either; Obi Wan plays 49% of his games with Colony compared to 16% when playing random cards iirc? That's a very different game indeed, imo, and it doesn't make for reasonable comparisons between different players, which is why it's weird to me to have it included on a leaderboard, much like the Larrys and the Karumahs etc.
There are technical solutions to the problem.

For example, you, I, someone could run a ranking system where over-sampled games for a given player are corrected for by decreasing their weight.  Since Colony games count for 49% of Obi Wan's games, and they should only be 16%, then each Colony game updates his rating by only 16/49 of the ordinary update.

http://en.wikipedia.org/wiki/Importance_sampling

First, my opinion on the topic: While I would prefering playing "truely random", and a "truely random" leaderboard would sound like the right thing for the top of it, I think that for the lower regions of the leaderboard some kind of bias should still be possible for games that count for the leaderboard. You want to get matched to people that are "approximately your skill", therefore games have to be rated, but maybe you don't want to start with the full 150 or so cards, but bias to Base or Intrigue.
Also for the mid-regions, maybe you want to get some more games "you like", while getting ranked properly to find the "right" opponents.

Importance sampling could correct the automatch bias, when you know how it works, but it can not correct for manually declining games, because you don't have the data on which games are declined, and even if you had them you would need some kind of model to predict the "probability that a game with cards c1,...,c10 will be declined, given historical decline data.".
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: greatexpectations on April 24, 2012, 10:55:43 am
Importance sampling could correct the automatch bias, when you know how it works, but it can not correct for manually declining games, because you don't have the data on which games are declined, and even if you had them you would need some kind of model to predict the "probability that a game with cards c1,...,c10 will be declined, given historical decline data.".

if you had the data, you could model it off of the veto mode data dougz posted here (http://forum.dominionstrategy.com/index.php?topic=594.msg8195#msg8195). i would guess that games are likely to be declined or vetoed for similar reasons.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: Fabian on April 24, 2012, 10:58:04 am
greatexpectations, I'm of quite the opposite opinion. I think knowing how to take advantage of veto mode strategy is a competitive edge (though I'd much prefer if veto mode didn't exist, fwiw), whereas "figuring out" that playing 58% Colony games instead of 19% is good is not a competitive edge, it's playing a different game than everyone else.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: rrenaud on April 24, 2012, 10:59:30 am
When I use the built in features of the website to maximize my chances of winning, it's okay.

When you do it, it's not.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: Fabian on April 24, 2012, 11:02:37 am
Was that directed at me? Seems a bit harsh? As far as I know, everyone plays with veto mode, and consequently everyone makes choices on which cards to veto. Not everyone plays 58% Colony games though, I'm pretty sure.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: DStu on April 24, 2012, 11:05:07 am
if you had the data, you could model it off of the veto mode data dougz posted here (http://forum.dominionstrategy.com/index.php?topic=594.msg8195#msg8195). i would guess that games are likely to be declined or vetoed for similar reasons.

OK, with large enough data you might even come around that data and or estimate it given the games actually played. But I don't know if the samle size is large enough, for example if I decline combination (X,Y) against lower level opponents only, I don't think you can tell out of the data out of even 10k games if (X,Y) happend by chance only half as often as the should, or if it is because I declined.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: WanderingWinder on April 24, 2012, 11:05:25 am
Was that directed at me? Seems a bit harsh? As far as I know, everyone plays with veto mode, and consequently everyone makes choices on which cards to veto. Not everyone plays 58% Colony games though, I'm pretty sure.
Certainly not everyone plays with veto mode. I understand where rrenaud is coming from, and I also get where you're coming from. I think you're both making valid points.

Not that this doesn't relate to OWB, but at this point, could we get this forked over to isotropic discussion? I'd like to make some more points on the topic, but it does feel a little weird to post them here.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: theory on April 24, 2012, 11:14:53 am
Not that this doesn't relate to OWB, but at this point, could we get this forked over to isotropic discussion? I'd like to make some more points on the topic, but it does feel a little weird to post them here.
Done.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: rrenaud on April 24, 2012, 11:25:54 am
I did direct it at you.

I personally play with the default, opp can choose veto mode if he wants to.  In veto games I just turbo pick random.

I certainly agree that vetoing cards that cause too much randomness or first player advantage is a lot more subtle and skill-intensive than just knowing that colonies help the better players.  Also, there has been discussion from the high level players about what veto strategies they use.

But they still look the same in the abstract, it's using the features of the site to bias the kingdoms to where you think you have an advantage.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: timchen on April 24, 2012, 11:53:00 am
This discussion pops up from time to time... I think I have transited to the phase "unless I can get paid by my level on the leaderboard, I don't care."

But a side note on veto mode:
I have changed my thoughts. The marginal benefit as discussed before is pretty small. The real effect is that veto mode benefits BM-ish player. Killing a card from 11 to kill a potential engine is much easier than killing a money strategy.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: olneyce on April 24, 2012, 12:11:27 pm
I get where people are coming from in the desire for total randomness.  I just don't fully agree.  Playing with veto mode makes the game more enjoyable for some people.  I used to set it as required for the first month or so it was out, but stopped because I didn't like excluding myself from playing a lot of people.  But when it does show up for games now, I do still enjoy pondering the strategy a bit.

Similarly, I can totally see why people would find Colony games to simply be more enjoyable than non-Colony games. 

Given that, I don't really see much of a problem with people employing those things.  I'd change my tune if someone could demonstrate that the upper-bound on comparative advantage someone could gain from 'exploiting' these loopholes is actually quite large. But given that I can only imagine it bumps players up a couple levels, I just can't get too worked up about.

But then, I also think it would be fine to allow a player to set a couple restraints on cards they simply want to avoid.  I understand that it's not 'true' Dominion to avoid playing with Familiar and Possession.  But those cards are just so incredibly frustrating that I'd rather corrupt the game a tiny bit than have to deal with them.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: greatexpectations on April 24, 2012, 12:20:04 pm
the desire for total randomness is related to competitive play.  i don't think anyone here has said that biasing or veto mode make the game less fun.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: jonts26 on April 24, 2012, 12:22:40 pm
I think the way the ranking is currently structured is great for 99.9% of players. If the purpose is to give players a general sense of how strong they are and match them up with similarly skilled players, it does a great job. If the purpose is to give a player a sense of improvement as he moves up in rankings, it does a good job. The only place where it becomes problematic is at the very top, where we want this ideal of who's the best. But really, the way the ranking system is structured is too broad anyway for separating players of very close skill. We use the (Mean - 3 STD) formula for level which is kind of silly when you have some people with standard deviations points lower than others just because they play more. I'm not saying we should abide cheaters like Karumah or Paralyzed (and it was probably right to restructure the rankings after Paralyzed), but whether or not one uses biases or veto mode, well what's a few levels among friends?
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: Captain_Frisk on April 24, 2012, 12:48:30 pm
Crazy thought: since it seems that we all agree that colony games vs. province games are different beasts... what if we just just had 2 separate leader boards?
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: paddyodoors on April 24, 2012, 01:15:36 pm
I get where people are coming from in the desire for total randomness.  I just don't fully agree.  Playing with veto mode makes the game more enjoyable for some people.  ...

Similarly, I can totally see why people would find Colony games to simply be more enjoyable than non-Colony games. ...

With respect, olneyce, I don't think this is really addressing the question at hand.  I don't think people are making points regarding whether veto mode or biases should be present and available on isotropic, rather whether games employing these things (which are extraneous to the game itself) should be part of the objective ranking system of the leaderboard.

What is the point of a leaderboard or a ranking system if it isn't our best approximation of skill?  For the purposes of this discussion, the key word here is "best."  How do we, as a community, best approximate each player's skill in Dominion generally.  It is not a question of preferences.  Or at least it shouldn't be if we're talking about something objective (i.e. not relative) like the current leaderboard seems to be attempting.

If OWB (or Paralyzed or Karumah or Fabian or WW or whomever) happens to be the leader at a given time, I think it should mean something.  If a player can make decisions outside of the scope of the game itself (i.e. Dominion by Donald X.) and influence his/her ranking, then it damages the integrity of the honor of being the top player.  It strips the leaderboard of meaning.  Certainly not ALL meaning, but, just as certainly, still strips it of some.

========

I am not a top player.  With all honesty, I will probably never be -- and that's okay.  All of these questions/comments/discussions about "30+", "40+", "how to be at the top", etc. will most likely never apply to me.  For my part, of course I play for fun.  If someone is not here (or on iso) to have fun, I hereby respectfully declare that person, IMHO, an idiot.

And I, personally, don't need the leaderboard to have fun, but it still adds an additional flavor for those of us with a competitive streak (which is cool).  And so, IF the leaderboard is going to be there, I want it to stand for something.  And I'm just a lowly peon -- I can only imagine what others whose names are close to the top think (except when they post here  :P).

If someday I ever do make it to the top (don't hold your breath), I want it to be because I was weighed and measured objectively, and found to be the best.

========

The best criticism of my position is this: "who is to say that full-on random is the most objective way to play Dominion?  Nowhere in any of the boxes published by RGG does it define pure Dominion vs. impure Dominion."  And the assertion in this criticism is 100% factual.

... to this I would reply that, by definition, any game which allows the introduction of any sort of player preference outside of the scope of the game itself isn't truly objective. (i.e. preferences and objectivity are mutually exclusive.)

(edit: sorry for the text-wall.  I doubt I make friends when I do this... :()
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: paddyodoors on April 24, 2012, 01:18:46 pm
Crazy thought: since it seems that we all agree that colony games vs. province games are different beasts... what if we just just had 2 separate leader boards?

That is, of course, another route that could be taken.  However, I suspect that the distinction between Province games and Colony games is like a difference of species but not of genus.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: Jfrisch on April 24, 2012, 01:23:22 pm
I think people are forgetting that the primary purpose of the ranking system is to give a rough approximation of skill in order for similarly skilled players to play each other. The ranking system, as is, has larger imperfections when used as a metric to judge true ability. (note that for people who are active and have played 1000+ games, the current subtraction of 3 times the standard deviation is not useful as a measure of skill).

Beyond this, my impression is that people don't (or, perhaps more accurately ,shouldn't) judge the skill of top players solely by ranking but rather by their skill, judgement, and general ability to win. WW is a better player than me because he's more likely to beat me.  (we have an 8/11 record which granted isn't that bad, but still I feel like he outplays me more than the record implies). Rankings have enough inherent flaws that other means (for example, tournaments) are a far better way to determine skill at top levels.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: blueblimp on April 24, 2012, 01:36:43 pm
I'd like to challenge the implicit assumption that choosing among all cards uniformly at random is the most legitimate way to pick a Dominion kingdom. The rules say "players can choose the 10 Kingdom cards using any method they agree on." Choosing uniformly at random makes some cards really awful, such as Scout and Transmute, which are both better in games dominated by their expansion. It puts just as much weight towards weak cards as strong cards, which favours money strategies.

I'd argue that Colony games are generally agreed to be underrepresented by isotropic's default kingdom generation. In the Dominionstrategy.com tournament, 5 of the 9 pre-designed kingdoms selected for the final had Colonies. Surely this means that the kingdom designers and judges feel that playing about 50% Colony games is actually a good way to determine player skill.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: Thisisnotasmile on April 24, 2012, 01:40:59 pm
I'd like to challenge the implicit assumption that choosing among all cards uniformly at random is the most legitimate way to pick a Dominion kingdom. The rules say "players can choose the 10 Kingdom cards using any method they agree on." Choosing uniformly at random makes some cards really awful, such as Scout and Transmute, which are both better in games dominated by their expansion. It puts just as much weight towards weak cards as strong cards, which favours money strategies.

I didn't really want to have any input into this discussion because I thought the way it started was pretty distasteful, but I just want to say that this is 100% correct, and the key words are "any method they agree on". If I agree to play a game chosen at random from one copy of every card in the game and you put 3 more copies of a handfull of cards in the the randomiser without telling me, the set has not been picked legitimately because I did not agree to those cards being added.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: Robz888 on April 24, 2012, 01:45:28 pm
I don't fault Obi Wan for figuring out a way to play more of the type of games that he enjoys, that he is best at, while still having them count for the leaderboard. The problem is, now that we all know he is doing that, we sort of have the incentive to do the same thing for whichever sets we are best at (Council Room has this information, if you don't know.) For instance, I guarantee that I could bias Intrigue and Hinterlands and give myself a competitive advantage. And other players can do the same for Alchemy or Cornucopia or whatever cards they are awesome with.

So if he's doing it, we should all do it, and if we all do it, ugh. So I would support not counting biased games for the leaderboard in the future.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: olneyce on April 24, 2012, 01:59:51 pm
Rankings have enough inherent flaws that other means (for example, tournaments) are a far better way to determine skill at top levels.
Hmm. Your ideas are intriguing to me and I wish to subscribe to your newsletter.

But seriously, I understand that people are concerned about the desire for rankings to reflect objective skill.  I'm just saying that these things (veto, biasing) insert a very small amount of noise into an intrinsically very noisy metric.  And since they increase enjoyment for a fair number of people, it doesn't seem worth worrying about very much.

The alternative, where the games stop counting for rankings creates a forced choice.  Players who enjoy biasing or vetoing but ALSO enjoy having a metric that assesses their rank are now obligated to put those values into conflict.

If this were something that really MATTERED, I would choose fairness over enjoyment for sure.  But it's just a game, and it's supposed to be fun.  Making it less fun for a lot of players so that the rankings are marginally more accurate just doesn't seem like a worthwhile trade to me.  It may very well seem worthwhile to others, and that's perfectly reasonable.  I just don't think the majority (or anything close to it) would share that belief.

That said, if the change were implemented, I don't think it would be catastrophic. 

And I tend to agree that if you ARE playing random, you shouldn't be able to see the kingdom.  Random ought to actually mean random.  But even there I'm not overly worried about the damage it does to the objectivity of rankings so much as I am annoyed at having people take forever to ponder kingdoms while deciding whether to accept on auto-match.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: Jfrisch on April 24, 2012, 02:18:58 pm
Certain players find certain Kingdom's unfun though. I personally would rather not play a kingdom which will devolve into a possession/destroy your deck game. Even games which I often will play, I am occasionally not in the mood for, Sea Hag/BM slogs can be frustrating and if I only have a limited amount of time to play I'd rather play a kingdom which looks interesting. I understand that being excessively choosy about kingdoms often keeps people from seeing certain aspects of the game, but I suspect the majority of top players here are uninterested in playing Warehouse/TM games, and it's not obvious to me why games which feel (and, to be fair, are) swingy to us can be derided, while games which feel swingy to newer players (i.e. possession, tournament, sea hag) should be suffered through.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: Deadlock39 on April 24, 2012, 02:21:02 pm
My 2 cents...

I agree with the assertion that a truly accurate leaderboard should be comparing only people who are playing by the same rules.  If you consider two hypothetical extremes, a player who only plays Colony games, and a player who only plays Province games, it wouldn't really make any sense to try to rank them against each other.  (They can't even play head to head.) 

There is clearly a difference between Colony and Province games in Dominion, but is it significant enough to have separate leaderboards?  This might depend on the player.  There is pretty clear evidence that individuals can have differing levels of skill in each.  Now, if there is a distinction between Province and Colony, you could also argue that there are cards that warp the game as much or more than adding Colonies does (Goons, Cursing, ect?) so do we rank those separately too? 

I think it would be interesting to try to normalize all of the parameters as was suggested by rrenaud would be interesting.  This could even take things further to account for cards like Goons, or cards that some players will decline to play with.  If you play tons of Colony games, then they are weighted less, so that their overall weight is the same an average unbiased game.  If you decline every game with Possession, then you have a hole in your ranking (higher uncertainty?) where the ~7% of games that should have contained it belong.  Play only games with King's Court/Goons/Masquerade... Well then there is a hole in your ranking where the 99.9% of games that should not have included the combo belong.

Everyone should be entitled to play the game the way they enjoy it.  OWB shouldn't be pressured to play less Colony games just because most players don't play that way, and his skill shouldn't be unrecognized by a leaderboard, but agree with Fabian that he is "playing a different game than everyone else" that he is being ranked against.

Regarding playing random boards... It is simply the easiest way to create a common environment for everyone to play in.  I think it would be really cool to see a setting that would use the card combo/counter idea that has recently been discussed in a couple threads.  It seems like that would guarantee a more interesting Kingdom much more often.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: paddyodoors on April 24, 2012, 02:21:26 pm
Rankings have enough inherent flaws that other means (for example, tournaments) are a far better way to determine skill at top levels.
Hmm. Your ideas are intriguing to me and I wish to subscribe to your newsletter.
rofl  ;D ;D ;D
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: toaster on April 24, 2012, 03:08:32 pm
Was that directed at me? Seems a bit harsh? As far as I know, everyone plays with veto mode, and consequently everyone makes choices on which cards to veto. Not everyone plays 58% Colony games though, I'm pretty sure.

I deliberately avoid veto mode.  I only ever play with it when I forget to select "prohibit veto mode" and hit accept before I notice the error.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: greatexpectations on April 24, 2012, 03:20:07 pm
But seriously, I understand that people are concerned about the desire for rankings to reflect objective skill.  I'm just saying that these things (veto, biasing) insert a very small amount of noise into an intrinsically very noisy metric.  And since they increase enjoyment for a fair number of people, it doesn't seem worth worrying about very much.

The alternative, where the games stop counting for rankings creates a forced choice.  Players who enjoy biasing or vetoing but ALSO enjoy having a metric that assesses their rank are now obligated to put those values into conflict.

If this were something that really MATTERED, I would choose fairness over enjoyment for sure.  But it's just a game, and it's supposed to be fun.  Making it less fun for a lot of players so that the rankings are marginally more accurate just doesn't seem like a worthwhile trade to me.  It may very well seem worthwhile to others, and that's perfectly reasonable.  I just don't think the majority (or anything close to it) would share that belief.

That said, if the change were implemented, I don't think it would be catastrophic. 

And I tend to agree that if you ARE playing random, you shouldn't be able to see the kingdom.  Random ought to actually mean random.  But even there I'm not overly worried about the damage it does to the objectivity of rankings so much as I am annoyed at having people take forever to ponder kingdoms while deciding whether to accept on auto-match.

man, i like that you are friendly and civil but i disagree with a lot of what you have to say. 

a few things:
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: DStu on April 24, 2012, 03:26:47 pm
Quote from: http://dominion.isotropic.org/faq/
What do the numbers mean?
    In the grand scheme of things, not much. Play for fun.
Okay, I promise not to take it too seriously. But really, how do I interpret the numbers?
...

I think one point is we have the same method for both "casual" players and the top of the board. And for everything except the top having some small deviation from "true random" is probably a good idea. If we really care about the top, maybe someone should run a tournament?
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: jonts26 on April 24, 2012, 03:29:51 pm
I really don't see a tournament as a great way to judge "best" in much the same way that baseball playoffs don't always have the best team winning it all. Dominion has so much randomness that you need a ton of games to get a meaningful sample size. If we did a tournament, we'd have to do like best out of 101 or something.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: rrenaud on April 24, 2012, 03:32:24 pm
I am skeptical that basing inferences about top players from a couple hundred games in a tournament will do a better job than basing inferences on the tens of thousands of games they play in general.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: jonts26 on April 24, 2012, 03:48:14 pm
I am skeptical that basing inferences about top players from a couple hundred games in a tournament will do a better job than basing inferences on the tens of thousands of games they play in general.

Which was pretty much my point. A tournament would need a huge sample size to be relevant, but we already have a huge sample size from the current leaderboard.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: elahrairah13 on April 24, 2012, 04:03:52 pm
Maybe under the bias options there should be a checkbox for "no bias". if you have that checked, you only play non-bias games. and so automatch won't put you with people who are checking bias boxes

I think there's nothing wrong with having the biased games on the leaderboard. (although it would be kinda cool if the leaderboard let you filter by various things)
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: rrenaud on April 24, 2012, 04:04:50 pm
Perhaps somewhat ironically, the best way to get unbiased games is to bias towards everything.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: O on April 24, 2012, 04:56:46 pm

Guys.. doesn't rating only unbiased games bias the ratings towards people who play well with a relatively even distribution of sets? I mean, Donald X created the game in expansions, theres a reasonable argument to be made that games are generally meant to be played with cards from 1-3 sets, as is the norm in real life play?

The idea of an objective, canonical dominion system for selecting cards is rather inane. Nowhere in the rules of dominion does it say "pick 10 randomizers from the entire set of randomizers from every set in existence".
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: Thisisnotasmile on April 24, 2012, 05:05:46 pm

Guys.. doesn't rating only unbiased games bias the ratings towards people who play well with a relatively even distribution of sets? I mean, Donald X created the game in expansions, theres a reasonable argument to be made that games are generally meant to be played with cards from 1-3 sets, as is the norm in real life play?

The idea of an objective, canonical dominion system for selecting cards is rather inane. Nowhere in the rules of dominion does it say "pick 10 randomizers from the entire set of randomizers from every set in existence".


No, it says to pick ten in any way that all players agree to, and as such I believe that only games picked in a fashion that all players agreed to should be included. If I bias towards Prosperity and my opponent biases towards Prosperity, it should count. If I select no bias and my opponent selects no bias, it should count. If I select no bias and my opponent biases towards Prosperity, it should not count (or in an ideal world we would not even be matched up at all).
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: jonts26 on April 24, 2012, 05:09:02 pm

Guys.. doesn't rating only unbiased games bias the ratings towards people who play well with a relatively even distribution of sets? I mean, Donald X created the game in expansions, theres a reasonable argument to be made that games are generally meant to be played with cards from 1-3 sets, as is the norm in real life play?

The idea of an objective, canonical dominion system for selecting cards is rather inane. Nowhere in the rules of dominion does it say "pick 10 randomizers from the entire set of randomizers from every set in existence".

Nowhere in the rules does it stipulate that a ranking leaderboard should exist at all. People keep bringing up rules for card selection but that seems a non issue to me here. The leaderboard can be structured however we (or just DougZ) feels it should be structured.

That said, you bring up the idea of playing expansions more grouped together. And I think that would be a fine way to force ranked games to be played. Just the same as full random. BUT the issue is not grouping expansions, rather only biasing towards one expansion which accents your strengths. If I biased a different expansion every day, I don't think anyone would argue the validity of my ranking, as I would still be getting an even sampling of all of dominion.

TL,DR: The ideal is that the leaderboard is a measure of how good you are at all of dominion, not just one set.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: O on April 24, 2012, 05:15:22 pm

Guys.. doesn't rating only unbiased games bias the ratings towards people who play well with a relatively even distribution of sets? I mean, Donald X created the game in expansions, theres a reasonable argument to be made that games are generally meant to be played with cards from 1-3 sets, as is the norm in real life play?

The idea of an objective, canonical dominion system for selecting cards is rather inane. Nowhere in the rules of dominion does it say "pick 10 randomizers from the entire set of randomizers from every set in existence".

Nowhere in the rules does it stipulate that a ranking leaderboard should exist at all. People keep bringing up rules for card selection but that seems a non issue to me here. The leaderboard can be structured however we (or just DougZ) feels it should be structured.

That said, you bring up the idea of playing expansions more grouped together. And I think that would be a fine way to force ranked games to be played. Just the same as full random. BUT the issue is not grouping expansions, rather only biasing towards one expansion which accents your strengths. If I biased a different expansion every day, I don't think anyone would argue the validity of my ranking, as I would still be getting an even sampling of all of dominion.

TL,DR: The ideal is that the leaderboard is a measure of how good you are at all of dominion, not just one set.

Right. My point was that about 10% of our (most of us) games on isotropic should count, because we play this bizarre way where you rarely get more than 3 cards from an expansion.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: jonts26 on April 24, 2012, 05:22:59 pm

Guys.. doesn't rating only unbiased games bias the ratings towards people who play well with a relatively even distribution of sets? I mean, Donald X created the game in expansions, theres a reasonable argument to be made that games are generally meant to be played with cards from 1-3 sets, as is the norm in real life play?

The idea of an objective, canonical dominion system for selecting cards is rather inane. Nowhere in the rules of dominion does it say "pick 10 randomizers from the entire set of randomizers from every set in existence".

Nowhere in the rules does it stipulate that a ranking leaderboard should exist at all. People keep bringing up rules for card selection but that seems a non issue to me here. The leaderboard can be structured however we (or just DougZ) feels it should be structured.

That said, you bring up the idea of playing expansions more grouped together. And I think that would be a fine way to force ranked games to be played. Just the same as full random. BUT the issue is not grouping expansions, rather only biasing towards one expansion which accents your strengths. If I biased a different expansion every day, I don't think anyone would argue the validity of my ranking, as I would still be getting an even sampling of all of dominion.

TL,DR: The ideal is that the leaderboard is a measure of how good you are at all of dominion, not just one set.

Right. My point was that about 10% of our (most of us) games on isotropic should count, because we play this bizarre way where you rarely get more than 3 cards from an expansion.

I think you missed my point a little, or I didn't state it clear enough. Ideally, to measure one's skill at dominion, you need to play with an even distribution of all cards. Thinking that full random is somehow more bizarre than grouped random is a little bizarre to me. Donald never even meant to break up all the cards into expansions originally. There were just cards.

Now there might be some small effect from playing grouped random, as some cards may change in power very slightly (scout?). But the effect has to be so minimal as to be lower than the noise already inherent in the leaderboard. I guess you can still make an argument either way, but it's going to be really weak.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: chwhite on April 24, 2012, 06:33:44 pm

Guys.. doesn't rating only unbiased games bias the ratings towards people who play well with a relatively even distribution of sets? I mean, Donald X created the game in expansions, theres a reasonable argument to be made that games are generally meant to be played with cards from 1-3 sets, as is the norm in real life play?

The idea of an objective, canonical dominion system for selecting cards is rather inane. Nowhere in the rules of dominion does it say "pick 10 randomizers from the entire set of randomizers from every set in existence".


No, it says to pick ten in any way that all players agree to, and as such I believe that only games picked in a fashion that all players agreed to should be included. If I bias towards Prosperity and my opponent biases towards Prosperity, it should count. If I select no bias and my opponent selects no bias, it should count. If I select no bias and my opponent biases towards Prosperity, it should not count (or in an ideal world we would not even be matched up at all).

Well, in that case I hereby agree to playing games with whatever biases my opponents wish to introduce into the randomization algorithm.  If they want to go Prosperity-heavy, or Hinterlands-heavy, or everything-but-Alchemy-heavy, or even base-heavy if they really want to, that's officially fine by me. I'll accept any games that aren't obvious setups (i.e. forcing specific cards)*.  And I won't bias myself unless there's a new expansion out and I want to play with the new cards.

*This is not quite true; I'll almost always decline KC-Goons-Masq games and the like if I catch it, and sometimes consider declining Amb/Masq-Possession matches on similar grounds.  If these two tiny exceptions taint my ranking in anybody's eyes, then I am quite willing to live with that.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: Obi Wan Bonogi on April 24, 2012, 08:06:00 pm

I agree with most of everything said so far especially what WanderingWinder said about me playing perfectly and the realistic top end for this game.  I think playing "perfectly" could get level 60 easy.  I'm honestly suprised Im #1.  I still constantly experiment and often feel like I dont know what the correct line is.  On a regular basis: I make huge blunders and throw games, take lines where if I had to do the board over I would try something different, play distracted by TV, streams, reddit, trees, etc and even tragically misclick.  And those are just the errors I know about. 

Obviously bias-Prosperity has helped me get high on the leaderboard I would never deny that, but that was never my intention.  I just enjoy Colony games and have enjoyed them since the day Prosperity was released.  Once Territory is released and costs 15g for 14vp that will surely become my favorite format is well.  I love engines and I love options.  I don't think Dominion would've held my interest as long as it has without the Colony-style game. 

I do think the learning curve on colony is lessening as more and more players in the high 30s and 40s are playing Colony games better so any windfall ranking advantage I have right now will deplete over time.  However, at all ranks there is an inherent advantage versus a lesser ranked opponent when playing colony boards.  That is to say with opponents 10 levels apart the higher ranked opponent has a better winrate on Colony boards.  Personally, I think this is an essential trait for a well-designed game to have.

That said, I obviously understand the gripes of people who don't bias-prosperity and feel lessened by their ranking.   There are some very strong colony players at the top end of the leaderboard that IF they started to bias-prosperity their winrate against lvl 39ish opponents would increase enough that they would pass me in level. 

If there was an option to play Colony-only games ranked on a Colony Leaderboard I would certainly love to play that format against like-minded opponents.  I the meantime I would suggest either not caring about levels because they dont mean much OR proving me right and using bias-prosperity and enough skill and focus to get to lvl 60...its possible.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: RisingJaguar on April 24, 2012, 09:16:42 pm
...
Maybe this deserves a whole other post, but what makes Colony games so different (and special)? 

I ask this sincerely.  Maybe I don't pay attention or my mind adapts by itself, but I don't see a huge difference between these games. 

Also, is there a reason why colony games only occur with Prosperity cards? Aside from that being an official rule and all, is there something about that set that NEEDS colony/platinum?
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: rrenaud on April 24, 2012, 09:17:46 pm
I think Colony games are just more rewarding to engines, and it's harder to put together an engine well than it is to play big money.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: blueblimp on April 24, 2012, 09:38:38 pm
Also, Colony games tend to be longer. Longer games mean more decisions. More decisions means more opportunities for the better player to make better decisions.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: Young Nick on April 24, 2012, 09:43:13 pm
The reason I have always assumed that P/C were in Prosperity was due to the high-cost cards. Spending $8 for a Forge or Expand is a lot less enticing when there are not C/P to ensure that you have a longer game and enough plays of your new, shiny, expensive card. This applies for any card >$6, in my opinion. I still think Colony games favor engines, if not for the fact that BMU + X is even MORE boring when the game lasts several turns longer.

I still think a relevant discussion is biasing any given set, not just Prosperity, and the potential outcome of this. I agree with TINAS about parallel constraints being grounds for a ranked game.

I also would like to see more discussion regarding the re-weighting of certain cards/combos so that everything is weighted to matter as much as it would come out on average. Thus, OWB's Colony games would matter somewhat less, and the KC/Goons/Masq wh**es' games without the combo would matter significantly more than the pin games. Is this possible? How would these weighting affect the current leaderboard?
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: O on April 24, 2012, 09:47:57 pm
Parallel constraints is a terrible idea; the constraints stack (I believe). So you can't have a singly biased ranked game under that system.

I honestly think that the ratings are fine as they are. We already have implicit agreement on biases because you either get to look at the set beforehand or veto the cards you don't like. So Obi Wan Bonogi either played people who were fine/happy with colony and prosperity games or vetoed the cards when they came up.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: DStu on April 25, 2012, 12:58:31 am
On a few random points:

While I don't have great problems with the biases, this:
I honestly think that the ratings are fine as they are. We already have implicit agreement on biases because you either get to look at the set beforehand or veto the cards you don't like. So Obi Wan Bonogi either played people who were fine/happy with colony and prosperity games or vetoed the cards when they came up.
is wrong. If you would  want to have unbiased games, you can not force that by denying Colony-games. You want 20% Colonygames. You don't know if your opponent has a bias. You don't get matched to them so often that you can see it. So there is no way to you no correct a bias.

@ColoniesInNonProsperityGames:
I never actually liked the rule that the probability of Colonies is proportional to the number of Propsperity cards. All sets maybe except base support strong engines that are capable of reaching Colonies as well as Prosperity is. And even with just Basecards you can reach $11. It does not really matter if I want to build my engine out of Worker's Village+Rabble or Farming Village+Margrave. Or if I trash with Trade Route/Loan or Salvager.

@reweighting:
From the data we have at the moment it's pretty difficult I would say. Always biasing Prosperity is a simple thing, because you have 2 cards that are influenced quite heavily, namely Colony/Province. Always biasing one other set should also be possible from the number of games with each card, though it's more difficult. Especially if it's Alchemy/Hinterlands/Cornucopia. Hinterlands/Cornucopia are released a lot later than isotropic, so there will naturally be less numbers of games played with these cards.  Alchemy has less numbers because it had the "0 or 3++ cards rule" implemented for a while. But how many less? That depends on how much you played then and between and now. So you can not only look on the numbers of games played with the card of each set, but have to take care of the date you played these games also.  But we have this data, so it should still be possible.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: Ratsia on April 25, 2012, 02:30:26 am
However, at all ranks there is an inherent advantage versus a lesser ranked opponent when playing colony boards.  That is to say with opponents 10 levels apart the higher ranked opponent has a better winrate on Colony boards.
Translated to the TrueSkill terminology, this would imply that the beta-parameter (controlling the probability of winning given a certain skill-difference) should be smaller for colony boards. However, quantifying the effects of that to the overall leaderboard is not so straightforward.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: timchen on April 25, 2012, 02:50:45 am
I think Colony games are just more rewarding to engines, and it's harder to put together an engine well than it is to play big money.
In addition, in colony games there is almost never a situation when one can edge out a win when the other player spent too much time building too much. (I mean sure if you insist it is possible, but it is a mistake that is a lot harder to make.)

The reason behind this phenomenon is just that in lots of colony games the platinum is such a great card.
And it make drastic difference to your buying power, which makes the colony buying decision a lot more obvious.

Compared to a money oriented province game,  here the winning play is to start buying provinces with uncertainty.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: Geronimoo on April 25, 2012, 05:02:19 am
Province vs Colony or how Silver is broken...

Silver is maybe the most efficient card in the game: you only pay $3 for a card that will give you $2 each time you play it AND there is no restriction to playing it because it's a Treasure. A hand like 3 Silver, 2 Copper is super easy to aquire very fast and = Province .The power of Jack is due mostly to the power of Silver. When we get to Colony games there is no all Silver hand that'll allow you to buy Colony, so a deck can not rely on Silver anymore and needs to branch out.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: paddyodoors on April 25, 2012, 08:58:52 am
Certainly not everyone plays with veto mode. I understand where rrenaud is coming from, and I also get where you're coming from. I think you're both making valid points.

Not that this doesn't relate to OWB, but at this point, could we get this forked over to isotropic discussion? I'd like to make some more points on the topic, but it does feel a little weird to post them here.

WW, are you still looking to make more points on the topic, as you said?  I am interested to hear your thoughts.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: WanderingWinder on April 25, 2012, 09:05:09 am
Certainly not everyone plays with veto mode. I understand where rrenaud is coming from, and I also get where you're coming from. I think you're both making valid points.

Not that this doesn't relate to OWB, but at this point, could we get this forked over to isotropic discussion? I'd like to make some more points on the topic, but it does feel a little weird to post them here.

WW, are you still looking to make more points on the topic, as you said?  I am interested to hear your thoughts.
Yep. Busy couple days, long post. Will try to do when I get home.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: WanderingWinder on April 25, 2012, 11:44:03 am
So, the thing with the ranking system - what is it that you want it to do? You want it to tell you the relative strengths of the different players at the game. But what is the game? This is the issue. To a certain extent, every single different board is a different game. So, somebody's probably the best in the world at playing 'the first game' somebody else at set Y, etc. etc. etc. But those don't come up that often, so you want to have a ranking that's somewhat independent of the board. I mean, I think that ideally, you have different ratings for different system. I have a prosperity-bias rating, a full random rating, maybe a preset-kingdom rating, a veto mode rating, etc. etc. I should certainly have different ratings for different numbers of players in the game - somebody who's really good at 2-player might not be so good at 4-player. I think you have different ratings to reflect this. Now, the isotropic system tries to combine all these things into some kind of overall rating. You could look at having the averaged out ratings of all possible boards, or something like that, as an overall rating anyway. But the question is what to include here, and how much to weight different things. Now, some people, let's say many people, have the idea that the game is really about how to react to any board, with no restrictions, yadda yadda yadda, and that should be totally random. That if you're not playing with attacks, you're missing a big part of the game. That if you're manipulating the kingdom in any way at all, you're shifting off of the overall, average-out skill that they think rating should reflect. I agree with them, but it's not that man, they're obviously right. It's that that's what I want out of my rating system. Different people want different things, and that's fine. So anyway, I think there ought to be different rating systems, and then people are just wanting different things out of the same couple numbers, and you just can't make 2 numbers be 12 different things. Ok.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: WanderingWinder on April 25, 2012, 11:59:24 am
Now the specific issue of OWB/biasing prosperity. First off, biasing prosperity does more than just add more colony games. It also gets you those cards more often, so I'd guess he knows them more than other people, better than other people, and that gives him a bit of an advantage. It also helps 2- and 3-card combos to hit more, which helps too; you recognize them, know how to play them, know their strengths. So like, I could bias intrigue all the time, and I'd know how to use the intrigue cards better, yadda yadda yadda yadda. So anyway, it inflates the winrate even when colonies aren't present.
Ok.
Now, the idea that colonies don't make a big difference is pretty laughable to me. The idea that they just help the engine player is a little off, but largely right. The idea that they make games longer and that just helps the better player, well, that's pretty messed up. It promotes different things. It promotes a different style of gameplay. It promotes different strategies and combinations. Not always ones that are harder to do, not always does it reduce variance, not always does it help the stronger player. It just doesn't. I mean, platinum and colony have consistently consistently been in my bottom 5 win rate given availability cards for a very long time. Now, I guess this might just mean that I suck, and I get really lucky all the time in the non-colony games. But I don't think so. It's just a different skillset.
Ok, now for OWB-specific stuff. Even if we assume that the only difference is playing with colony and platinum, which as I point out above, isn't true, then let's look at these numbers. Pulling the data up today... He has 4393 games, overall winrate of 1.27. 2566 games with platinum and colony, winrate when those are available is 1.32. Okay, this means that there are 1827 games without platinum and colony, and his winrate in these games would be approximately (can't calculate exactly because of rounding) 1.20. If we scale that down to 19% (I can't tell what the exact percentage should be, because for some reason the availabilities for your basic cards aren't the same in the overall data), we'd get a winrate of 1.22. Which is a pretty big difference. I can't tell exactly how that affects his rating, but it's going to knock it back down a couple levels, at least, I expect. But as we all know, he's a top player anyway. Just not the clear #1 that the board makes it look like (unless you think that the current rating system take everything into account totally perfectly yadda yadda yadda see my last post).
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: WanderingWinder on April 25, 2012, 12:40:30 pm
Oh, also, I don't think this is going to be a big deal for iso. I mean, what are you going to do with all these games that we have? It's a big issue without a real solution. But I hope that the new official server has different policies here, as in having more rating pools, having games where you can't see the cards before you accept, having something for full random, etc. etc.
I hope it also has a spectator mode, where you can watch other people playing. These are my two biggest wishlist items for the official app in comparison to what we have on iso.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: ecq on April 25, 2012, 12:56:06 pm
Province games are shorter and simple strategies tend to be more competitive.  Big Money + X wins a lot of the time, and a level 0 who goes first and picks the correct X can give a top-ranked player a run for his money on a healthy percentage of boards.

Colony games have a lower percentage of boards where simple Big Money + X wins.  Further, Prosperity contains several cards that would make you favor at least Big Money + X + Y, if not a full-blown engine, so simpler strategies aren't as competitive.  Weaker players have fewer chances to cause an upset, so stronger players would have fewer rank-damaging losses.

That's my guess, at least.

More importantly, though, I think skill comparisons based on level are pretty meaningless among the top players.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: paddyodoors on April 25, 2012, 01:10:06 pm
... More importantly, though, I think skill comparisons based on level are pretty meaningless among the top players.

Good point... I never thought of that, but it seems true.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: olneyce on April 25, 2012, 07:13:12 pm
man, i like that you are friendly and civil but i disagree with a lot of what you have to say. 

a few things:
  • you are right, we don't know exactly how much influence biasing and veto will have on levels. but i'm not sure we can dismiss it as being negligible either. by my math, OWB wins 6% more colony games than he does province only games and he makes it so that about 60% of his games have been colony games. we can argue on whether a 6% change is a "small amount of noise" or a large amount, but it is something.  and i'm also not sure i would call TrueSkill a "very noisy metric". 
  • i still don't think the fun aspect should matter as much for competitive play.  but if you want to bring it up, just remember that it can cut both ways. i prefer playing with a fully random set. no automatic vetoing of attack cards.  no bias towards sets. i prefer the random mix of games, and i think that the current IsoDom challenge helps demonstrate this.  how many match reports in the first round were basically hey there's witch/BM again.  ditto masquerade for intrigue.  should some people enjoying a bias give them preference over my preference for a fully random set?  and that is before we even consider if that bias gives them any advantage at all.  winning games is going to be more fun than losing most of the time, so any change which cuts into that is not something i will be eager to embrace.
  • it is a game and it is supposed to be fun.  but that doesn't mean that we can just cast aside the competitive aspect of it.  chess and baseball are just games too. fun should always be a factor, but you can't enter in to a competitive environment and expect that people aren't going to care about advantages.  if you play in a softball beer league yeah everyone there is trying to have a good time but they are still gonna get mad if they catch you trying to sneak a lead from first base.
Yeah, I don't really agree with this, but I totally get where you're coming from.

The main point I'd make is that (for the most part) you already get the type of games that you want, and you get a ranking system that reflects performance over time.  The lack of a truly-random option is unfortunate, but if you set it to no-veto and just bias for ALL of the sets you'll get something fairly close to random.

A world where bias or veto games stop counting for ranking would impose a far more strict constraint on people who feel otherwise. 

As for competition vs. fun, I clearly agree that there needs to be a (fairly high) baseline of competitive balance.  And I absolutely would reject anything that would interfere with the internal structure of the game (like a handicap for low-ranked players or something).  But the unofficial/official rankings on Isotropic?  Sure, it's important that they genuinely reflect a general level of skill and performance.  But I just can't get too worked up about them missing the 'true' level of someone by a few slots.

And I do think it is a pretty noisy metric.  Am I the third best player or the 35th?  Within the course of about two weeks I was in both slots.  I don't think my skill actually changed much there.  I mean, in the grand scheme of 10,000 players, that's still fairly precise.  But we're having a conversation about the very top and how people compare against each other there, right?
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: timchen on April 25, 2012, 08:58:11 pm
Is "yadaa" the same as "you know" or "blah" or "etc"?
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: rrenaud on April 25, 2012, 09:05:38 pm
I'd say yadda yadda yadda ~= blah blah blah, maybe it slightly connotates more bored and less agitated.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: WanderingWinder on April 25, 2012, 09:42:52 pm
Is "yadaa" the same as "you know" or "blah" or "etc"?
Basically yes.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: WanderingWinder on April 25, 2012, 09:47:54 pm
And I do think it is a pretty noisy metric.  Am I the third best player or the 35th?  Within the course of about two weeks I was in both slots.  I don't think my skill actually changed much there.  I mean, in the grand scheme of 10,000 players, that's still fairly precise.  But we're having a conversation about the very top and how people compare against each other there, right?
There's two major things going on here that cause these massive fluctuations. First, you're looking at the ranking which is based on level. Really, TrueSkill gives you a ranking that ought to be based off of the rating. And because you're uncertainty rises every day, whether you play or not... this has a big impact. Also, if you look at this range of the list, and in fact, lots of the list, everybody is pretty well bunched up. That people are relatively similarly-skilled does not necessarily mean that the skill rankings are wrong. They could well be right - if the people are relatively similarly-skilled. Which, to me, it seems they are.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: WanderingWinder on April 25, 2012, 09:49:22 pm
you are right, we don't know exactly how much influence biasing and veto will have on levels. but i'm not sure we can dismiss it as being negligible either. by my math, OWB wins 6% more colony games than he does province only games and he makes it so that about 60% of his games have been colony games. we can argue on whether a 6% change is a "small amount of noise" or a large amount, but it is something.  and i'm also not sure i would call TrueSkill a "very noisy metric".
Actually, you're underestimating things here. See post #80 for the maths, yo.
Title: Re: Discussion on non-random game selection and TrueSkill ranking
Post by: greatexpectations on April 25, 2012, 10:49:34 pm
Actually, you're underestimating things here. See post #80 for the maths, yo.

not underestimating, just different.  i think we have the same numbers we just used different methods and wording.

 i went and converted the win rates to a number of wins and then calculated a % of games won instead of using the 1.xx win rates.  i got 60% for province and 66% for colony.  that is a 6% difference as i tried to describe it.  convert that to win-rates and you get 1.20 and 1.32, same as you got.