Dominion Strategy Forum

Please login or register.

Login with username, password and session length
Pages: [1] 2 3  All

Author Topic: Ranking, Seeding, and Skill Discussion  (Read 24260 times)

0 Members and 1 Guest are viewing this topic.

Insomniac

  • Jester
  • *****
  • Offline Offline
  • Posts: 785
  • Respect: +392
    • View Profile
Ranking, Seeding, and Skill Discussion
« on: December 07, 2012, 12:10:25 pm »
0

There's also the matter of some players like myself who play IRL more often than on Iso.  I mean, when I entered this contest I hadn't played on iso since probably September.  When I signed back in my level was down to 12, although I guess playing a few games got it up to 18 by the time seeding was determined.

As I said during the discussion about wildcard entries, I disagree with the ISO seed allowing high entries, I mean I guess it's better than nothing, but I don't think of myself as a bad player, but I mean I NEVER play on Isotropic except when these tourneys are going down, otherwise I play in RL or heaven forbid on goko.
« Last Edit: December 09, 2012, 12:40:24 am by greatexpectations »
Logged
"It is one of [Insomniacs] badges of pride that he will bus anyone, at any time, and he has done it over and over on day 1. I am completely serious, it is like the biggest part of his meta." - Dsell

soulnet

  • Mountebank
  • *****
  • Offline Offline
  • Posts: 2142
  • Respect: +1751
    • View Profile
Re: Re: Upsets
« Reply #1 on: December 07, 2012, 01:01:41 pm »
0

When we have our Dominion Professional Tour rankings, we can use that to seed tournaments. In the meantime, Iso ranking is a pretty well approximation, with a small number of outliers (although they may be really important outliers, I must say).

On the opposite side, I would have been fine with pure random seeding, but watching two of the best players clashing in round 1 or 2 instead on an exciting semifinal or something its kind of sad.

Seeding has minor impact in Swiss system, but I guess the total number of games on a Swiss tournament is too big to handle (Challonge has Swiss brackets, so calculating the matchup should be no problem).
Logged

popsofctown

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5477
  • Respect: +2860
    • View Profile
Re: Re: Upsets
« Reply #2 on: December 07, 2012, 01:49:12 pm »
0

Games are simultaneous in Swiss though, and iso isn't going to crash because lots of people are on. 

We really don't have a good reason not to be using Swiss, besides easier administration (which, when that administration is being given on a volunteer basis I can't complain about)
Logged

theory

  • Administrator
  • *****
  • Offline Offline
  • Posts: 3603
  • Respect: +6121
    • View Profile
    • Dominion Strategy
Re: Re: Upsets
« Reply #3 on: December 07, 2012, 01:51:38 pm »
0

Games are simultaneous in Swiss though, and iso isn't going to crash because lots of people are on. 

We really don't have a good reason not to be using Swiss, besides easier administration (which, when that administration is being given on a volunteer basis I can't complain about)

Administration has a cost from a user perspective.  Swiss systems are better for something like IsoDom, with a small group of committed players.
Logged

popsofctown

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5477
  • Respect: +2860
    • View Profile
Re: Re: Upsets
« Reply #4 on: December 07, 2012, 01:55:59 pm »
0

I don't see a link between player commitment and tournament system.  Whichever player makes first will play about as many games as if it was Swiss.  Players who do not do well would be arranged into more games, but if those players are only interested in first place they can concede by PM. 
Logged

soulnet

  • Mountebank
  • *****
  • Offline Offline
  • Posts: 2142
  • Respect: +1751
    • View Profile
Re: Re: Upsets
« Reply #5 on: December 07, 2012, 03:34:24 pm »
0

You can use Swiss without simultaneous plays. I don't think revealing previous results will make you want to lose or tie to change your next play, but even then, results may be kept hidden until the end of the week of the round.

By "too big to handle" I was thinking on the people running the tournament. 128 games on Isotropic is not too bad, and also, as I said, they do not need to be simultaneous.

Maybe Swiss with some time to forfeit in between rounds? That way, people who lost interest due to not performing as good may withdraw without giving a semi-random set of people free points and one game less.
Logged

WanderingWinder

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5275
  • ...doesn't really matter to me
  • Respect: +4381
    • View Profile
    • WanderingWinder YouTube Page
Re: Re: Upsets
« Reply #6 on: December 07, 2012, 05:40:08 pm »
0

There's also the matter of some players like myself who play IRL more often than on Iso.  I mean, when I entered this contest I hadn't played on iso since probably September.  When I signed back in my level was down to 12, although I guess playing a few games got it up to 18 by the time seeding was determined.

As I said during the discussion about wildcard entries, I disagree with the ISO seed allowing high entries, I mean I guess it's better than nothing, but I don't think of myself as a bad player, but I mean I NEVER play on Isotropic except when these tourneys are going down, otherwise I play in RL or heaven forbid on goko.
Funny, I do sorta consider myself a bad player...

greatexpectations

  • Board Moderator
  • *
  • Offline Offline
  • Posts: 1097
  • Respect: +1067
    • View Profile
Re: Re: Upsets
« Reply #7 on: December 07, 2012, 05:57:39 pm »
+2

Funny, I do sorta consider myself a bad player...

Level 43   51.721 ± 8.108   20   10486   WanderingWinder

if you are bad then the 7919 of us lower down on the leaderboard must just be hideously awful.
Logged
momomoto: ...I looked at the tableau and went "Mountebank? That's for jerks."
rrenaud: Jerks win.

WanderingWinder

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5275
  • ...doesn't really matter to me
  • Respect: +4381
    • View Profile
    • WanderingWinder YouTube Page
Re: Re: Upsets
« Reply #8 on: December 07, 2012, 06:00:22 pm »
0

Funny, I do sorta consider myself a bad player...

Level 43   51.721 ± 8.108   20   10486   WanderingWinder

if you are bad then the 7919 of us lower down on the leaderboard must just be hideously awful.
Well, two things. One, please please please use mean skill, not level.
Two, I was originally going to make some complaint about Insomniac saying that he doesn't consider himself a bad player, because everyone thinks they are a good player - self-reporting bias! So i thought to myself 'who considers himself a bad player', and I was like, gee, I sorta do.

GigaKnight

  • Moneylender
  • ****
  • Offline Offline
  • Posts: 169
  • Respect: +54
    • View Profile
Re: Re: Upsets
« Reply #9 on: December 07, 2012, 07:05:04 pm »
0

Funny, I do sorta consider myself a bad player...

Level 43   51.721 ± 8.108   20   10486   WanderingWinder

if you are bad then the 7919 of us lower down on the leaderboard must just be hideously awful.
Well, two things. One, please please please use mean skill, not level.
Two, I was originally going to make some complaint about Insomniac saying that he doesn't consider himself a bad player, because everyone thinks they are a good player - self-reporting bias! So i thought to myself 'who considers himself a bad player', and I was like, gee, I sorta do.

Sorry, but what's "mean skill"?  If it's something specific and well-known, I'm not Googling well.  Or do you just mean something like "for the amount you play, you should be better"?

More-specifically, what qualities about yourself or your play would lead you to sorta consider yourself a bad player?  As a player I look up to, I'm wondering if you have any insights I can learn from.

Logged

greatexpectations

  • Board Moderator
  • *
  • Offline Offline
  • Posts: 1097
  • Respect: +1067
    • View Profile
Re: Re: Upsets
« Reply #10 on: December 07, 2012, 07:41:22 pm »
0

Well, two things. One, please please please use mean skill, not level.
Two, I was originally going to make some complaint about Insomniac saying that he doesn't consider himself a bad player, because everyone thinks they are a good player - self-reporting bias! So i thought to myself 'who considers himself a bad player', and I was like, gee, I sorta do.

mean skill only changes that number by maybe 20-25 people, the overall point still stands. and it was (i thought clearly) meant as a joke, there is really no need to get defensive. if there is any doubt, i will turn it into a first world problems meme next time. i fully understand your intent, just realize that to some extent a comment like that from a person who has hit lvl 50 and the #1 spot is a bit dismissive of those of us a bit further down the rankings.

Sorry, but what's "mean skill"?  If it's something specific and well-known, I'm not Googling well.  Or do you just mean something like "for the amount you play, you should be better"?

mean skill is the first (leftmost) number you will see on the leaderboard. it is what trueskill thinks your level is. there is a degree of uncertainty there, and this uncertainty is subtracted from the mean skill to determine your level on the leaderboard. there is a little more detail here if you are interested.
Logged
momomoto: ...I looked at the tableau and went "Mountebank? That's for jerks."
rrenaud: Jerks win.

GigaKnight

  • Moneylender
  • ****
  • Offline Offline
  • Posts: 169
  • Respect: +54
    • View Profile
Re: Re: Upsets
« Reply #11 on: December 07, 2012, 10:15:17 pm »
0

Sorry, but what's "mean skill"?  If it's something specific and well-known, I'm not Googling well.  Or do you just mean something like "for the amount you play, you should be better"?

mean skill is the first (leftmost) number you will see on the leaderboard. it is what trueskill thinks your level is. there is a degree of uncertainty there, and this uncertainty is subtracted from the mean skill to determine your level on the leaderboard. there is a little more detail here if you are interested.

Ah, thanks!  I knew about the levels and uncertainty; I just didn't realize it was also called "mean skill".

But now I'm confused why WW puts 3 pleases before asking you to use it instead of level.  :)  Is it vastly preferred in some circles?

EDIT: I should say I did read the Wiki and Iso FAQ (again) and saw that the Wiki said there was some debate, but I'm not familiar with that debate.  Is there a thread talking about why TrueSkill is/isn't better than TrueSkill + Uncertainty?
« Last Edit: December 07, 2012, 10:29:36 pm by GigaKnight »
Logged

greatexpectations

  • Board Moderator
  • *
  • Offline Offline
  • Posts: 1097
  • Respect: +1067
    • View Profile
Re: Re: Upsets
« Reply #12 on: December 07, 2012, 10:24:35 pm »
0

Ah, thanks!  I knew about the levels and uncertainty; I just didn't realize it was also called "mean skill".

But now I'm confused why WW puts 3 pleases before asking you to use it instead of level.  :)  Is it vastly preferred in some circles?

i am going to oversimplify here, but basically mean skill is a more accurate measure for skill for people on the top end of the leaderboard (or who have a ton of games played) and the isotropic level is a better measure for less skilled or newer players.
Logged
momomoto: ...I looked at the tableau and went "Mountebank? That's for jerks."
rrenaud: Jerks win.

GigaKnight

  • Moneylender
  • ****
  • Offline Offline
  • Posts: 169
  • Respect: +54
    • View Profile
Re: Re: Upsets
« Reply #13 on: December 07, 2012, 10:33:46 pm »
0

Ah, thanks!  I knew about the levels and uncertainty; I just didn't realize it was also called "mean skill".

But now I'm confused why WW puts 3 pleases before asking you to use it instead of level.  :)  Is it vastly preferred in some circles?

i am going to oversimplify here, but basically mean skill is a more accurate measure for skill for people on the top end of the leaderboard (or who have a ton of games played) and the isotropic level is a better measure for less skilled or newer players.

I see the high-level reasoning there.  Shouldn't that be easily tunable with the Isotropic parameters?  I mean, couldn't it further reduce uncertainty as you increase in level / games played so that it was accurate at either end?  This is purely academic, I suppose, but I guess I'm just not understanding WW's seemingly-passionate preference of TrueSkill.  As you pointed out earlier, it doesn't seem to make too too much of a difference.
Logged

WanderingWinder

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5275
  • ...doesn't really matter to me
  • Respect: +4381
    • View Profile
    • WanderingWinder YouTube Page
Re: Re: Upsets
« Reply #14 on: December 07, 2012, 11:02:52 pm »
0

Ah, thanks!  I knew about the levels and uncertainty; I just didn't realize it was also called "mean skill".

But now I'm confused why WW puts 3 pleases before asking you to use it instead of level.  :)  Is it vastly preferred in some circles?

i am going to oversimplify here, but basically mean skill is a more accurate measure for skill for people on the top end of the leaderboard (or who have a ton of games played) and the isotropic level is a better measure for less skilled or newer players.
That's patently false. The mean skill is the best guess you have for the skill of anyone. BY DEFINITION.

WanderingWinder

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5275
  • ...doesn't really matter to me
  • Respect: +4381
    • View Profile
    • WanderingWinder YouTube Page
Re: Re: Upsets
« Reply #15 on: December 07, 2012, 11:03:44 pm »
0

Well, two things. One, please please please use mean skill, not level.
Two, I was originally going to make some complaint about Insomniac saying that he doesn't consider himself a bad player, because everyone thinks they are a good player - self-reporting bias! So i thought to myself 'who considers himself a bad player', and I was like, gee, I sorta do.

mean skill only changes that number by maybe 20-25 people, the overall point still stands. and it was (i thought clearly) meant as a joke, there is really no need to get defensive. if there is any doubt, i will turn it into a first world problems meme next time. i fully understand your intent, just realize that to some extent a comment like that from a person who has hit lvl 50 and the #1 spot is a bit dismissive of those of us a bit further down the rankings.
Man, that's the opposite of my point. My point is not to take yourself too seriously.

SirPeebles

  • Cartographer
  • *****
  • Offline Offline
  • Posts: 3249
  • Respect: +5459
    • View Profile
Re: Re: Upsets
« Reply #16 on: December 07, 2012, 11:20:22 pm »
0

Ah, thanks!  I knew about the levels and uncertainty; I just didn't realize it was also called "mean skill".

But now I'm confused why WW puts 3 pleases before asking you to use it instead of level.  :)  Is it vastly preferred in some circles?

i am going to oversimplify here, but basically mean skill is a more accurate measure for skill for people on the top end of the leaderboard (or who have a ton of games played) and the isotropic level is a better measure for less skilled or newer players.
That's patently false. The mean skill is the best guess you have for the skill of anyone. BY DEFINITION.

You could say it's the most accurate guess.  To say that it's the best guess really depends on what you're trying to accomplish with your guess.
Logged
Well you *do* need a signature...

WanderingWinder

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5275
  • ...doesn't really matter to me
  • Respect: +4381
    • View Profile
    • WanderingWinder YouTube Page
Re: Re: Upsets
« Reply #17 on: December 07, 2012, 11:23:48 pm »
0

Ah, thanks!  I knew about the levels and uncertainty; I just didn't realize it was also called "mean skill".

But now I'm confused why WW puts 3 pleases before asking you to use it instead of level.  :)  Is it vastly preferred in some circles?

i am going to oversimplify here, but basically mean skill is a more accurate measure for skill for people on the top end of the leaderboard (or who have a ton of games played) and the isotropic level is a better measure for less skilled or newer players.
That's patently false. The mean skill is the best guess you have for the skill of anyone. BY DEFINITION.

You could say it's the most accurate guess.  To say that it's the best guess really depends on what you're trying to accomplish with your guess.
What are you trying to accomplish with your guess?

greatexpectations

  • Board Moderator
  • *
  • Offline Offline
  • Posts: 1097
  • Respect: +1067
    • View Profile
Re: Re: Upsets
« Reply #18 on: December 07, 2012, 11:28:49 pm »
0

That's patently false. The mean skill is the best guess you have for the skill of anyone. BY DEFINITION.

really dude? coming across a little harsh here. i know full well what mean skill is. i even purposefully left off the word 'skill' in my point about newer players, so i'm not actually sure that it is 'patently false'.

but that is beside the point. as i said in my post, i was trying to give gigaknight a 2 second explanation as to why mean skill is more useful at the top of the leaderboard vs the bottom. with newer players your uncertainty is so large that the mean skill is less valuable than it is otherwise. if anything i said was that far off then by all means i will change it. but i wasn't trying to be super specific, i just gave him a quick response and pointed him to where he could read up a little more.
Logged
momomoto: ...I looked at the tableau and went "Mountebank? That's for jerks."
rrenaud: Jerks win.

Stealth Tomato

  • Minion
  • *****
  • Offline Offline
  • Posts: 507
  • Dorkneel
  • Respect: +480
    • View Profile
Re: Re: Upsets
« Reply #19 on: December 08, 2012, 12:48:18 am »
+1

Funny, I do sorta consider myself a bad player...

Level 43   51.721 ± 8.108   20   10486   WanderingWinder

if you are bad then the 7919 of us lower down on the leaderboard must just be hideously awful.
Haha, most of us have learned to compare our skill to our peak skill--therefore those of us like WW are looking back at ourselves a year ago and going "Damn, where did all my skills go?"
Logged

Piemaster

  • Apprentice
  • *****
  • Offline Offline
  • Posts: 260
  • Respect: +170
    • View Profile
Re: Re: Upsets
« Reply #20 on: December 08, 2012, 12:18:26 pm »
+2

There are a million things that affect people's Iso level, and even mean skill, that have nothing to do with how good a player they are.

I mean when I play on Iso, I am often doing something else at the same time, sometimes only really playing attention when I get an audio alert.  And sometimes I quit games because I have to go out or something.  And some games I spot odd combos and think 'hmmm, I wonder if that will work'.  And then it doesn't and I lose.  I am currently level 24.  If I concentrated fully on every game and only played when I knew it would have my full attention and turned the TV off in the background and always played the strategy I thought would give me the highest % chance to win, then maybe I would be level 28 or even 30.  But would I actually be a better player?  No, in fact never trying anything new, interesting or fun could make me worse.
Logged

Mic Qsenoch

  • 2015 DS Champion
  • *
  • Offline Offline
  • Posts: 1709
  • Respect: +4329
    • View Profile
Re: Re: Upsets
« Reply #21 on: December 08, 2012, 12:30:05 pm »
+3

If I concentrated fully on every game and only played when I knew it would have my full attention and turned the TV off in the background and always played the strategy I thought would give me the highest % chance to win, then maybe I would be level 28 or even 30.  But would I actually be a better player?

Yes, clearly.
Logged

rrenaud

  • Administrator
  • *****
  • Offline Offline
  • Posts: 991
  • Uncivilized Barbarian of Statistics
  • Respect: +1197
    • View Profile
    • CouncilRoom
Re: Re: Upsets
« Reply #22 on: December 08, 2012, 01:05:33 pm »
0

I'd worry about definition and models.

If you talk about the definition within the model, then you are granting the models assumptions.  You even seem to disagree with some of trueskills assumptions.  And if you don't grant the models assumption, then all bets are off.

Yes, the mean is a great summary of a normal distribution. 

I am pretty sure that I can come up with some distributions of player strengths/#games played so that mean - 2 * std dev is a better measure for finding the best player than the mean is when sampled according to those strengths. 
Logged

WanderingWinder

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5275
  • ...doesn't really matter to me
  • Respect: +4381
    • View Profile
    • WanderingWinder YouTube Page
Re: Re: Upsets
« Reply #23 on: December 08, 2012, 01:17:12 pm »
0

I'd worry about definition and models.

If you talk about the definition within the model, then you are granting the models assumptions.  You even seem to disagree with some of trueskills assumptions.  And if you don't grant the models assumption, then all bets are off.

Yes, the mean is a great summary of a normal distribution. 

I am pretty sure that I can come up with some distributions of player strengths/#games played so that mean - 2 * std dev is a better measure for finding the best player than the mean is when sampled according to those strengths. 
Sure, if you don't buy trueskill, you can probably get a better estimate. The only way mean - 2*stdev as a single number will be better than mean, though, is if you take a skewed distribution, in which case you may really do better by taking something like median. Of course, in any case, you will do better if you can include more numbers - so mean & st dev is better than just mean, because it gives some indication as to the shape of the distribution rather than just the position. Mean is certainly not enough to define playing strength - certainly someone who hasn't played very much is going to have much higher uncertainty in our estimate than someone who has played a lot. So we aren't very confident in that guess, but the point is that it's still the best guess. I mean, there's really no evidence that mean + 3 st dev (and they use 3 rather than 2, yes?) is a worse estimator than subtracting. Your best guess is your best guess regardless, it's just how much variation you expect around that guess. Of course, you can argue with the methodology which assigns the best guess to the position it currently does, but that's a separate issue.

Piemaster

  • Apprentice
  • *****
  • Offline Offline
  • Posts: 260
  • Respect: +170
    • View Profile
Re: Re: Upsets
« Reply #24 on: December 08, 2012, 02:13:35 pm »
0

If I concentrated fully on every game and only played when I knew it would have my full attention and turned the TV off in the background and always played the strategy I thought would give me the highest % chance to win, then maybe I would be level 28 or even 30.  But would I actually be a better player?

Yes, clearly.

No, not really clearly at all.  Maybe I didn't phrase the example very well, let's try wording the question in a different way.

Hypothetically let's say I started a new Iso account, played 200 games and tried my absolute best and played with no distractions.  At the end of the 200 games I have a level of 25.  Then I play another 200 games.  In these I goof around, try some fun combos and unconventional decks, play while doing other things, and otherwise play sub-optimally.  After that my level has dropped to 20.  Have I actually got worse at the game?  Sure, I am playing worse, but if I entered a tournament I could play back at my old 25 level again so at that tournament my level really doesn't reflect my ability.

Or to use a non-Dominion example, every year the Williams Sisters turn up to Wimbledon seeded a long way down because they don't really play their best game other than at the majors.  They could probably try harder in the tour events and ply more to get a higher seeding for the slams, but it wouldn't mean they were better players at the slams, they would just have a higher ranking.
« Last Edit: December 08, 2012, 02:17:12 pm by Piemaster »
Logged
Pages: [1] 2 3  All
 

Page created in 0.076 seconds with 21 queries.