Dominion Strategy Forum

Please login or register.

Login with username, password and session length
Pages: 1 2 [3] 4 5  All

Author Topic: Leaderboard change  (Read 28263 times)

0 Members and 1 Guest are viewing this topic.

Dominionaer

  • Young Witch
  • ****
  • Offline Offline
  • Posts: 141
  • Respect: +38
    • View Profile
Re: Leaderboard change
« Reply #50 on: November 25, 2011, 04:53:39 am »
0

Can the TrueSkill algorithm get enhanced with a factor for number of opponents? Or that the second - third - fourth ... game against same opponent changes less?
« Last Edit: November 25, 2011, 04:57:39 am by Dominionaer »
Logged

Davio

  • 2012 Dutch Champion
  • *
  • Offline Offline
  • Posts: 4733
  • Respect: +3327
    • View Profile
Re: Leaderboard change
« Reply #51 on: November 25, 2011, 06:36:45 am »
+1

How about using a Ladder system which resets every 30 (or 90) days to accommodate the more active players? Every player starts with 0 and just tries to work his way up.

Besides the ladder, the hall of fame persistent rankings can be maintained.

On the main page it can show something like:
Davio 14/33 or something with the first number being the ladder rank and the latter being the all-time rank.
Logged

BSG: Cagprezimal Adama
Mage Knight: Arythea

Lekkit

  • 2011 Swedish Champion
  • *
  • Offline Offline
  • Posts: 1253
  • Shuffle iT Username: Lekkit
  • Respect: +673
    • View Profile
Re: Leaderboard change
« Reply #52 on: November 25, 2011, 08:37:37 am »
0

On the main page it can show something like:
Davio 14/33 or something with the first number being the ladder rank and the latter being the all-time rank.

I like this idea.
Logged

guided

  • Jester
  • *****
  • Offline Offline
  • Posts: 940
  • Respect: +88
    • View Profile
Re: Leaderboard change
« Reply #53 on: November 25, 2011, 11:48:57 am »
0

I wouldn't be that bothered except I haven't been playing much the last month and this torched my rating just in time for tournament seeding. A second all-time leaderboard would be nice. But yeah, the FAQ has always said "Okay, I promise not to take it too seriously." My approach to leaderboard 'sploiters like Paralyzed has ever been to roll my eyes and pay no attention to them.
Logged

Donald X.

  • Dominion Designer
  • *****
  • Offline Offline
  • Posts: 5480
  • Respect: +22105
    • View Profile
Re: Leaderboard change
« Reply #54 on: November 25, 2011, 06:02:27 pm »
0

Can the TrueSkill algorithm get enhanced with a factor for number of opponents? Or that the second - third - fourth ... game against same opponent changes less?
The simplest thing to do here would be to only rank games played against random opponents. If you pick your opponent, that game doesn't count. I don't know if this would bother people or what.

TrueSkill can only rank you within a community of players. If you just play one person then it's only ranking you vs. them. And other than the "look at me" guy, there are no doubt people who just/mostly go on isotropic to play friends who then get inaccurately ranked.

The most accurate thing would be to only show a list of you and the people you've played. Then you don't get a single leaderboard though. You could instead only rank people who have played against N people who have played against N people, as a way to guess whether you're really part of the community or not. You could still beat that approach by making tons of dummy accounts and having them all play each other, and maybe this is a community where it's hard to get your first N games because people want to know your ranking in order to play you; dunno there.
Logged

Guy Srinivasan

  • Salvager
  • ****
  • Offline Offline
  • Posts: 61
  • Respect: +38
    • View Profile
Re: Leaderboard change
« Reply #55 on: November 25, 2011, 06:15:50 pm »
0

If you want a leaderboard, then this change is absolutely fine, but showing people their "ranking" when they're not in the top tier of players is silly - the leaderboard is optimized to show who's at the very tippy top. If you want a measure of skill, then using TrueSkill levels is a bit silly, since they're optimized for a leaderboard-like system. Otherwise subtracting uncertainty from mean skill as a "conservative measure of skill" doesn't make much sense to me. If I had my druthers ;) then it'd show the TrueSkill mean and some sense of the uncertainty like "Rating: 30 (28-32)" vs "Rating: 30 (27-33)".
Logged

Reyk

  • Young Witch
  • ****
  • Offline Offline
  • Posts: 146
  • Respect: +24
    • View Profile
Re: Leaderboard change
« Reply #56 on: November 26, 2011, 04:50:05 am »
0

The simplest thing to do here would be to only rank games played against random opponents.

The sad thing is that you have to pick your opponent after the new leaderboard was released. Because the auotmatch +/- level option no longer works in a meaningful way.
Logged

painted_cow

  • Moneylender
  • ****
  • Offline Offline
  • Posts: 151
  • Respect: +19
    • View Profile
Re: Leaderboard change
« Reply #57 on: November 26, 2011, 08:23:14 am »
0

The sad thing is that you have to pick your opponent after the new leaderboard was released. Because the auotmatch +/- level option no longer works in a meaningful way.

Furthermore why should I play say a level 13 shark_bait or level 27 theory??? It would be completely dumb for me to do it. I can only lose points in a long run...Not really encouraging. So, I am still waiting for the good old leaderboard, please set it back, dougz :-)
Logged

fellowmartian

  • Swindler
  • ***
  • Offline Offline
  • Posts: 19
  • Respect: +5
    • View Profile
Re: Leaderboard change
« Reply #58 on: November 26, 2011, 08:34:58 am »
+2

There seems to be a biggish groundswell of opinion to the effect that this is a bad idea. I said in a recent post elsewhere that you have to ask what a leaderboard is for. I think most people think of it as some combination of:

a) a challenge, a bit of fun to see how high you can go
b) a way to see how you are progressing in skill
c) a way to find similarly-skilled opponents.

The new system seems a significantly worse way to achieve all three of these. The only people who are going to be able to use it in this way are people who play very regularly. People who've been working their way up but have to have a break will see themselves drop away again; that's discouraging. People who've dropped away but are really good players won't show as such (if I want a level 27 and I get theory, (c) above is right out).

Add one more to the chorus of 'please change it back', or at least provide the two alternatives...

Or, if this is permanent, then... rrenaud, councilroom is separate, right? And uses all the same data? In which case would a separate leaderboard based around the old system be plausible on councilroom? Or something like this? We could put our councilroom levels in our statuses if needs be. I'm no techie so forgive me if this isn't a plausible idea.
« Last Edit: November 26, 2011, 10:28:17 am by fellowmartian »
Logged

Razzishi

  • Conspirator
  • ****
  • Offline Offline
  • Posts: 216
  • Shuffle iT Username: Eye Urn
  • Respect: +120
    • View Profile
Re: Leaderboard change
« Reply #59 on: November 26, 2011, 09:35:35 am »
0

I fail to see the point of this change.  Trueskill by itself already does whatever this change would possibly be intended to do if properly configured.  I think that the way of only adding to a player's uncertainty on days that they play games is a really strange thing to do, and that every player's base uncertainty should go up by a small amount each ranking period regardless of whether they play or not while playing games at a consistent skill level will naturally lower uncertainty.  That way people will slowly drift down the leaderboard when they're inactive, and Trueskill will work its magic moving them quickly to the rank that they should be when they return.

edit: You could then add a significant increase to every player's uncertainty when a new set comes out; I think that the changing nature of the game might have been a major part of rationale behind this change.
« Last Edit: November 26, 2011, 09:37:49 am by Razzishi »
Logged
Stop reading my signature.

popsofctown

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5383
  • Respect: +2769
    • View Profile
Re: Leaderboard change
« Reply #60 on: November 26, 2011, 01:36:56 pm »
0

Can the TrueSkill algorithm get enhanced with a factor for number of opponents? Or that the second - third - fourth ... game against same opponent changes less?
The simplest thing to do here would be to only rank games played against random opponents. If you pick your opponent, that game doesn't count. I don't know if this would bother people or what.

TrueSkill can only rank you within a community of players. If you just play one person then it's only ranking you vs. them. And other than the "look at me" guy, there are no doubt people who just/mostly go on isotropic to play friends who then get inaccurately ranked.

The most accurate thing would be to only show a list of you and the people you've played. Then you don't get a single leaderboard though. You could instead only rank people who have played against N people who have played against N people, as a way to guess whether you're really part of the community or not. You could still beat that approach by making tons of dummy accounts and having them all play each other, and maybe this is a community where it's hard to get your first N games because people want to know your ranking in order to play you; dunno there.

You should know, though.  It's not.  Come play with us Donald :(
Logged
Also you probably are an expert if you buy two bureaucrats early.

rspeer

  • Witch
  • *****
  • Offline Offline
  • Posts: 469
  • Respect: +875
    • View Profile
Re: Leaderboard change
« Reply #61 on: November 26, 2011, 03:33:53 pm »
0

CouncilRoom does have its own implementation of TrueSkill. I'm seeing if I can get a sensible approximation to the previous leaderboard out of it.

There is some missing information, unfortunately. We don't know whether the players were logged in or not, and we don't have unique IDs for their accounts -- so if a player changes their name, it would have to start their rating over.

Also, it's coming out in a different range somehow. Nobody ever ends up over level 30. Which might have been how the old old leaderboard worked. Does anyone remember what the change was that dougz made before this? I think it had to do with how uncertainty was added to ratings, such as adding the uncertainty factor once per day instead of once per game or something.
« Last Edit: November 26, 2011, 03:36:30 pm by rspeer »
Logged

theory

  • Administrator
  • *****
  • Offline Offline
  • Posts: 3594
  • Respect: +6035
    • View Profile
    • Dominion Strategy
Re: Leaderboard change
« Reply #62 on: November 26, 2011, 03:42:26 pm »
0

Also, it's coming out in a different range somehow. Nobody ever ends up over level 30. Which might have been how the old old leaderboard worked. Does anyone remember what the change was that dougz made before this? I think it had to do with how uncertainty was added to ratings, such as adding the uncertainty factor once per day instead of once per game or something.
Yeah.  He used to increase it a little bit per game, but no one could gain enough points to overcome that.  So now it's a little bit per any day that you play at least one game. 
Logged

guided

  • Jester
  • *****
  • Offline Offline
  • Posts: 940
  • Respect: +88
    • View Profile
Re: Leaderboard change
« Reply #63 on: November 26, 2011, 04:01:44 pm »
+1

I always wondered, what if it's per game but smaller? Or what if it's per day period but smaller?

The "per day that you play a game" has always subtly discouraged me from logging on if I don't have time to play a bunch of games.
Logged

nsnyder

  • Swindler
  • ***
  • Offline Offline
  • Posts: 15
  • Respect: +5
    • View Profile
Re: Leaderboard change
« Reply #64 on: November 26, 2011, 04:16:59 pm »
0

The uncertainty tweak never really made sense to me.  The reason the "variation" part was high before is just that Dominion is a high variation game.
Logged

nsnyder

  • Swindler
  • ***
  • Offline Offline
  • Posts: 15
  • Respect: +5
    • View Profile
Re: Leaderboard change
« Reply #65 on: November 26, 2011, 04:21:46 pm »
0

What I find frustrating about the change is that it used to be that I could look at how my rating changed to get an idea of how well I played the day before, but now it doesn't work that way because it's as much a reflection on how well I played 30 days ago as it is on how I played yesterday.  So it's actually less responsive to my play rather than more responsive.
Logged

rspeer

  • Witch
  • *****
  • Offline Offline
  • Posts: 469
  • Respect: +875
    • View Profile
Re: Leaderboard change
« Reply #66 on: November 26, 2011, 04:54:39 pm »
+3

I always wondered, what if it's per game but smaller? Or what if it's per day period but smaller?

The "per day that you play a game" has always subtly discouraged me from logging on if I don't have time to play a bunch of games.

Adding uncertainty per day makes a lot more sense to me.

I'm experimenting with this right now: at the start of each day (or for now, each 15,000-game set, because I'm too lazy to try to parse dates out of the councilroom data), everyone's variance moves 1% back toward its default value.

If you play frequently and constantly, this is equivalent to the previous leaderboard. If you play frequently and then don't play for a week, your uncertainty increases (and therefore your level decreases) by about 1.7 points. While you're not playing, your level effectively decays with a half-life of 69 days.

I'll see how it comes out. Although I would be mostly happy with just having the on-Isotropic leaderboard back the way it was.
Logged

popsofctown

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5383
  • Respect: +2769
    • View Profile
Re: Leaderboard change
« Reply #67 on: November 26, 2011, 05:29:25 pm »
0

Wait, is NO ONE over thirty?

Is it possible that my 3 rank drop doesn't mean I'm sucking lately?
Logged
Also you probably are an expert if you buy two bureaucrats early.

Rjax36

  • Herbalist
  • **
  • Offline Offline
  • Posts: 6
  • Respect: 0
    • View Profile
Re: Leaderboard change
« Reply #68 on: November 26, 2011, 06:12:13 pm »
0

I feel sorry for my opponents in this new system :(

I'm lifetime lvl 36 (I've topped off at #16 on the leaderboard) and I play tons of paper dominion even when I'm inactive online, so the 5-20 games I play each month should be pretty miserable for my opponent's ratings...

I STRONGLY prefer the old system :P
Logged

Axxle

  • Torturer
  • *****
  • Offline Offline
  • Posts: 1664
  • Most Valuable Serial Killer
  • Respect: +1950
    • View Profile
Re: Leaderboard change
« Reply #69 on: November 26, 2011, 06:37:15 pm »
0

I find that currently I'm ranked about the same and haven't seen many problems with it.  But I can definitely forsee myself trailing off in activity like before and would be very discouraged if I had to crawl back up from rank 0 again.  I'd be very bored with playing people much below my rank in order to do so.  So count me in for reverting the change.
Logged
We might be from all over the world, but "we all talk this one language  : +1 card + 1 action +1 buy , gain , discard, trash... " - RTT

Masticore

  • Pearl Diver
  • **
  • Offline Offline
  • Posts: 10
  • Respect: +2
    • View Profile
Re: Leaderboard change
« Reply #70 on: November 26, 2011, 07:21:12 pm »
0

So i think this is an interesting idea, but this way isn't giving it a chance. The time period is too small, it needs to be 3 months at absolute least, hopefully 6 months. For now i say change it back and then revise a plan to make it better
Logged

Lekkit

  • 2011 Swedish Champion
  • *
  • Offline Offline
  • Posts: 1253
  • Shuffle iT Username: Lekkit
  • Respect: +673
    • View Profile
Re: Leaderboard change
« Reply #71 on: November 26, 2011, 09:32:41 pm »
0

I was thinking about this earlier today, and I think that keeping levels and the leaderboard separate is the way to go. Levels could reflect overall stats, and the leaderboard could reflect recent achievments.

Generally I feel like people don't like the changes because their level doesn't reflect their skill. Keeping level on another board than the leaderboard would solve this problem, right?
Logged

Dominionaer

  • Young Witch
  • ****
  • Offline Offline
  • Posts: 141
  • Respect: +38
    • View Profile
Re: Leaderboard change
« Reply #72 on: November 27, 2011, 03:44:50 am »
0

Generally I feel like people don't like the changes ...
I like my jump up. And can lie to me, that those ahead are not out of reach.
Logged

Kirian

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 7092
  • Shuffle iT Username: Kirian
  • An Unbalanced Equation
  • Respect: +9371
    • View Profile
Re: Leaderboard change
« Reply #73 on: November 27, 2011, 02:04:23 pm »
+3

The sad thing is that you have to pick your opponent after the new leaderboard was released. Because the auotmatch +/- level option no longer works in a meaningful way.

Furthermore why should I play say a level 13 shark_bait or level 27 theory??? It would be completely dumb for me to do it. I can only lose points in a long run...Not really encouraging. So, I am still waiting for the good old leaderboard, please set it back, dougz :-)

Exactly!  Think of this from a game theory perspective:

No reasonable person of high skill and high knowledge will play against shark_bait while he's level 13, if they're interested in a maintaining a position on the leaderboard--that is, if the board has any interest to them as a little bit of competition.  So a few different types of people will play him:

People with no interest in using the board for competition (who wouldn't anyway)
People who don't know his old ranking
People with very low rankings

The first and third will be essentially unaffected.  Two groups are affected:  those who won't play him but would have been fine playing him at level 40+, and those who don't know that old ranking.  The former lose out on an interesting opponent; the latter group have their scores disproportionately deflated.  Meanwhile, because higher-ranked people won't play him, his rating won't rise as fast as it ought to, perpetuating the problem.

I think fellowmartian has it right above.  It serves none of the likely purposes of a leaderboard, and most definitely kills what I think is the main purpose--to be able to play against those nearest in skill to you.  There's even a menu option to force auto-match to do that for you!  But that menu option is now... much less useful.

I basically have two options:  ignore the leaderboard (all right, I can do that).  Or--and this is worse--not play for a week or so and strategically deflate my rating in an attempt to inflate it by playing games against people who ought to be much lower than I am.  I'm not nearly as well-known as shark_bait or theory (except, perhaps, among members of this board).  It's not my style, I won't do it... which means basically ignoring the leaderboard for mow, which saddens me a bit.

Doug?  You there man?  Tons of top players begging for this change to be reverted, not for the sake of their ratings but for the sake of the system.
Logged
Kirian's Law of f.DS jokes:  Any sufficiently unexplained joke is indistinguishable from serious conversation.

rspeer

  • Witch
  • *****
  • Offline Offline
  • Posts: 469
  • Respect: +875
    • View Profile
Re: Leaderboard change
« Reply #74 on: November 27, 2011, 03:56:09 pm »
+3

Here's an experimental leaderboard with the "uncertainty decay" thing I was talking about, where the uncertainty range gradually increases toward 25 at a rate of 1% per day. There are probably glitches.

http://councilroom.com:8888/leaderboard

(This link will probably go dead in a few days. Hopefully because the code gets merged into the main site.)

Interestingly, the top player on it is samojedi, who if I recall correctly was the top player on the very first leaderboard. I had no idea he was still playing.

Also interestingly, the level range is still smaller than isotropic's (not to say that that's inherently bad). I wonder what other unpublished tweaks Doug has made to the rating code.
Logged
Pages: 1 2 [3] 4 5  All
 

Page created in 0.102 seconds with 21 queries.