Dominion Strategy Forum

Please login or register.

Login with username, password and session length
Pages: 1 [2] 3 4 5  All

Author Topic: The WW Rating System  (Read 32732 times)

0 Members and 1 Guest are viewing this topic.

dondon151

  • 2012 US Champion
  • *
  • Offline Offline
  • Posts: 2522
  • Respect: +1857
    • View Profile
Re: The WW Rating System
« Reply #25 on: April 21, 2013, 06:49:23 pm »
+1

I think the point of the WW rating system is so that WW can be on top :D

EDIT: oh, I confused the player list with a tentative ranking.
« Last Edit: April 21, 2013, 07:09:51 pm by dondon151 »
Logged

WanderingWinder

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5275
  • ...doesn't really matter to me
  • Respect: +4389
    • View Profile
    • WanderingWinder YouTube Page
Re: The WW Rating System
« Reply #26 on: April 21, 2013, 07:06:47 pm »
+1

I think the point of the WW rating system is so that WW can be on top :D
This is ironic - right now, if we ran it, I would be pretty far in last.

markusin

  • Cartographer
  • *****
  • Offline Offline
  • Posts: 3846
  • Shuffle iT Username: markusin
  • I also switched from Starcraft
  • Respect: +2438
    • View Profile
Re: The WW Rating System
« Reply #27 on: April 21, 2013, 08:22:18 pm »
0

I'll sign up for this. I'd only be able to play at night, but I'm still in a position to contribute to this. My Goko username is markusin.

As for the rating system itself, I like how you've focused on functionality over prettiness, even if that means having negative numbers and rather large point differences. I understand that you're really going for precision and openness, but if you ever need to reduce the size of the ratings to make it more marketable you can just throw in a bracketing system to lump together ratings that are say 10 points apart. You'd keep the real ratings as is, but display the number of the bracket as the rating players see. Or is that the kind of thing that makes rating systems go horribly wrong?
Logged

serakfalcon

  • Apprentice
  • *****
  • Offline Offline
  • Posts: 278
  • Shuffle iT Username: serakfalcon
  • Respect: +230
    • View Profile
Re: The WW Rating System
« Reply #28 on: April 21, 2013, 08:53:19 pm »
0

Big numbers are harder to understand. So, small numbers are good if you want people to feel like their ranking has meaning, big numbers are fine if you deliberately don't.

Quote
There's a similar argument about negative numbers - if everything is positive... like someone with a 1400 chess rating can easily think 'aha! I am half as good as the best players in the world, since my rating is half theirs' - but of course this is wrong, because it's not on a ratio scale. But one of my hopes is that a negative number, or the commonality of such, will stop people from thinking this, because such thinking doesn't really make sense with negatives.
The only problem with this is you'll end up with this sort of perverse selection where people with negative scores may get discouraged and drop out of the system, because they 'aren't any good'. People really don't like having negative scores on anything. For now its probably ok to test, but once you have an idea of 'how low things can go' you may want to adjust that

Also, sign me up. My Goko username changes but usually something like its a (adjective) mangoose - (adverb)
« Last Edit: April 21, 2013, 08:54:36 pm by serakfalcon »
Logged

Qvist

  • Mountebank
  • *****
  • Offline Offline
  • Posts: 2400
  • Shuffle iT Username: Qvist
  • Respect: +4086
    • View Profile
Re: The WW Rating System
« Reply #29 on: April 21, 2013, 09:45:40 pm »
0

Sign me up, now reading the details.

timchen

  • Minion
  • *****
  • Offline Offline
  • Posts: 704
  • Shuffle iT Username: allfail
  • Respect: +235
    • View Profile
Re: The WW Rating System
« Reply #30 on: April 22, 2013, 01:24:28 am »
0

Hmm, the relation between the win rate and DR is interesting. For example if A beats B 2/3 of the time and B beats C 2/3 of the time as well, the relation will predict A beats C 4/5 of the time. Why is this the case?
Logged

DStu

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2627
  • Respect: +1490
    • View Profile
Re: The WW Rating System
« Reply #31 on: April 22, 2013, 02:00:14 am »
0

I think DStu has downloaded the all the goko logs and probably has a system set up to do so.

No, I only have the logs of one week, and the "system" is
Code: [Select]
curl dominionlogs.goko.com/20130412/ -o 20130412.dat
less 20130412.dat | grep -oE ">log.*txt" | tr -d '>' | xargs -I{} wget -nc dominionlogs.goko.com/20130412/{}

but this gives me a download rate of ~50kB/sec per job, so I if I want some logs I usually wait for some days and load a bunch of days at once.

Edit: You could easily get all the logs where some player hosted a game by their (let's call them) hash in the filename, but to get the other games I think you must download all the logs and search for the player name.

Edit2:
Code: [Select]
less ../20130412.dat | grep -oE ">log.*txt" | tr -d '>' | xargs -I{} echo http://dominionlogs.goko.com/20130412/{} | wget -nc -i -
this one is a bit faster (~125kB/sec)
« Last Edit: April 22, 2013, 02:23:46 am by DStu »
Logged

ftl

  • Mountebank
  • *****
  • Offline Offline
  • Posts: 2056
  • Shuffle iT Username: ftl
  • Respect: +1345
    • View Profile
Re: The WW Rating System
« Reply #32 on: April 22, 2013, 02:07:57 am »
0

A big problem is that the goko logs don't separate pro games and casual games. You can separate out bot games based on the names of the bots (though it probably takes some tedious looking through to get rid of all the bots from adventure mode...), but there's no way to tell pro or casual from the log yet.

Could be interesting to run the rating system on the logs just as a calibration to see what happens, but it wouldn't really be valid for people who played a lot of hand-picked games with friends, or who played a bunch of games with sets they're good at. Or who deliberately gamed casual mode at some point or another. 
Logged

serakfalcon

  • Apprentice
  • *****
  • Offline Offline
  • Posts: 278
  • Shuffle iT Username: serakfalcon
  • Respect: +230
    • View Profile
Re: The WW Rating System
« Reply #33 on: April 22, 2013, 04:40:11 am »
+1

Quote
Edit: You could easily get all the logs where some player hosted a game by their (let's call them) hash in the filename, but to get the other games I think you must download all the logs and search for the player name.

If you've got all the logs then you could just rate everyone on Goko, same idea as CR parsing I guess, just need some coding
Logged

lespeutere

  • 2012 German Champion
  • *
  • Offline Offline
  • Posts: 488
  • Respect: +390
    • View Profile
Re: The WW Rating System
« Reply #34 on: April 22, 2013, 04:48:22 am »
0

sign me in, pls.

Name's all capitalised on goko: LESPEUTERE
« Last Edit: April 22, 2013, 04:51:41 am by lespeutere »
Logged

Qvist

  • Mountebank
  • *****
  • Offline Offline
  • Posts: 2400
  • Shuffle iT Username: Qvist
  • Respect: +4086
    • View Profile
Re: The WW Rating System
« Reply #35 on: April 22, 2013, 06:57:22 am »
+1

I've read it now in detail. I'm looking forward for this although I have some doubts that you will get enough data as everything has to be done manually. Best would be if you only have to fill in (into a spreadsheet of form) the URL of the log and the rest will be done automatically (like ending conditions and stuff).

WanderingWinder

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5275
  • ...doesn't really matter to me
  • Respect: +4389
    • View Profile
    • WanderingWinder YouTube Page
Re: The WW Rating System
« Reply #36 on: April 22, 2013, 07:52:09 am »
0

serakfalcon: I need a specific username to tie you to in order to sign you up.
Also, I want people to have only the vaguest notion that a larger number is better than a smaller one, and nothing more (without consulting formula), so yeah, I think the big numbers serve me fine.

timchen, yes, this is simply the idea behind the logistic curve. You can google it, or I can try to give a thumbnail later today when I have more time. I do not believe it is correct for Dominion especially, and this is my biggest innovation, however it is what is widely used and as good a starting point as any - do you have some other suggestion?

Log Parsing: Man, if someone wants to do that, for everyone, it is fine by me. But I lack the resources and time.

Qvist, well, I don't expect to have tons of things. Something like 10 games yesterday, and I expect this might go up a bit, but not just a whole ton. The URL of the log would force me to write a log parser, and then make sure it has one of the right people, and there are a few other issues. And anyway, the only thing that is STRICTLY necessary for now is the two players and the result, so okay, I don't see this as that much harder than pasting a URL, especially since you have to go and get it. Anyhow, if people don't want to do that, I'm not THAT interested in getting this done anyway - it's okay to me. But if you want to write such a parser, then people can always use that to input their data.

lespeutere

  • 2012 German Champion
  • *
  • Offline Offline
  • Posts: 488
  • Respect: +390
    • View Profile
Re: The WW Rating System
« Reply #37 on: April 22, 2013, 09:27:56 am »
0

So I'm in Outpost II (w/o any cards). I've got to do some real stuff, too, so I'll check once every 15ish min whether someone's joined me.
« Last Edit: April 22, 2013, 10:11:34 am by lespeutere »
Logged

jonts26

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2746
  • Shuffle iT Username: jonts
  • Respect: +3671
    • View Profile
Re: The WW Rating System
« Reply #38 on: April 22, 2013, 01:53:33 pm »
0

Sign me up. I'm going by just jonts on goko. I'm not sure how much I'll actually play, but I'll be around here and there.
Logged

A Drowned Kernel

  • 2015 World Champion
  • *
  • Offline Offline
  • Posts: 1067
  • They/Them
  • Respect: +1981
    • View Profile
Re: The WW Rating System
« Reply #39 on: April 22, 2013, 03:02:43 pm »
0

I'd like to sign up, my goko name is A Drowned Kernel.
Logged
The perfect engine
But it will never go off
Three piles are empty

serakfalcon

  • Apprentice
  • *****
  • Offline Offline
  • Posts: 278
  • Shuffle iT Username: serakfalcon
  • Respect: +230
    • View Profile
Re: The WW Rating System
« Reply #40 on: April 22, 2013, 08:16:48 pm »
0

Quote
But if you want to write such a parser, then people can always use that to input their data.

I'm working on it now. I need to practice regex anyway for my own benefit, this at least gives me a reason to.

My user is "a mad mangoose - marvelously" I guess I'll keep it fixed for now.
Logged

lespeutere

  • 2012 German Champion
  • *
  • Offline Offline
  • Posts: 488
  • Respect: +390
    • View Profile
Re: The WW Rating System
« Reply #41 on: April 23, 2013, 03:12:29 am »
0

Just in case it could cause confusion: Which timezone should we adjust the entries to? If I play at 1 am, e.g., it's still the day before in the US.
Logged

Beyond Awesome

  • Global Moderator
  • *****
  • Offline Offline
  • Posts: 2941
  • Shuffle iT Username: Beyond Awesome
  • Respect: +2467
    • View Profile
Re: The WW Rating System
« Reply #42 on: April 23, 2013, 06:47:33 am »
0

I'm in. Goko name Beyond Awesome
Logged

WanderingWinder

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5275
  • ...doesn't really matter to me
  • Respect: +4389
    • View Profile
    • WanderingWinder YouTube Page
Re: The WW Rating System
« Reply #43 on: April 23, 2013, 08:03:34 am »
0

Just in case it could cause confusion: Which timezone should we adjust the entries to? If I play at 1 am, e.g., it's still the day before in the US.

This is something we can work out. It matters in a sense - your uncertainty will be slightly different tomorrow than it is today - but this difference is small. I personally don't care, as long as you don't end up having it on both days.

Davio

  • 2012 Dutch Champion
  • *
  • Offline Offline
  • Posts: 4787
  • Respect: +3413
    • View Profile
Re: The WW Rating System
« Reply #44 on: April 23, 2013, 08:04:57 am »
0

The idea behind a rating should be a way to predict the outcome of a match between players of a certain rating, right?

Would it be an idea to, besides listing players on a leaderboard ordered by their rating, to show personalized (generated) boards for every player? For instance, if I look at my own leaderboard it shows me in the middle with my rating and for every other player it shows my predicted win % against them instead of their rating. This might help people understand what's going on.
Logged

BSG: Cagprezimal Adama
Mage Knight: Arythea

WanderingWinder

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5275
  • ...doesn't really matter to me
  • Respect: +4389
    • View Profile
    • WanderingWinder YouTube Page
Re: The WW Rating System
« Reply #45 on: April 23, 2013, 08:11:01 am »
0

The idea behind a rating should be a way to predict the outcome of a match between players of a certain rating, right?
Yes, essentially.
Quote
Would it be an idea to, besides listing players on a leaderboard ordered by their rating, to show personalized (generated) boards for every player? For instance, if I look at my own leaderboard it shows me in the middle with my rating and for every other player it shows my predicted win % against them instead of their rating. This might help people understand what's going on.
I'm not sure what you are saying here - you want me to make a list like this and... PM it to everyone every morning? I don't have the time to write that many PMs? Theoretically I like it, but it's too much to do... Hmm, maybe if I can find the time at some point to write a little something that will show anyone their expected % in any particular matchup they play? I can probably do this eventually, though it would likely be at least the weekend before I get the time.

DStu

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2627
  • Respect: +1490
    • View Profile
Re: The WW Rating System
« Reply #46 on: April 23, 2013, 08:27:38 am »
0

While we are at it: Shouldn't the uncertainty play some role in the expected win rate?
Logged

Davio

  • 2012 Dutch Champion
  • *
  • Offline Offline
  • Posts: 4787
  • Respect: +3413
    • View Profile
Re: The WW Rating System
« Reply #47 on: April 23, 2013, 08:29:01 am »
0

No, what I meant was: Make it queryable.

So http://someurl/leaderboard/player=Davio shows the same players as the normal leaderboard but the ratings for the other players are expected win percentages instead of ratings, something like that at least.

Or is there any way to map regular ratings to percentages? That would make them more understandable, but this can only work with a bottom and top limit...
Logged

BSG: Cagprezimal Adama
Mage Knight: Arythea

lespeutere

  • 2012 German Champion
  • *
  • Offline Offline
  • Posts: 488
  • Respect: +390
    • View Profile
Re: The WW Rating System
« Reply #48 on: April 23, 2013, 09:40:54 am »
0

Might try to pretty up formatting lter.

Current Rating List



Rank ID Rating Uncertainty Name
1 6 12381 6700            Watno
2 10 6244 7299        ftl
3 13 3750 7425    lespeutere
4 4 2420 7256        Tables
5 3 534 7157         Twistedarcher
6 15 0 7500   A Drowned Kernel
7 14 0 7500   jonts26
8 12 0 7500   Qvist
9 8 0 7276       Schneau
10 7 0 7500      Kirian
11 2 0 7500      michaeljb
12 9 -2929 6987          qmech
13 11 -5108 7110          markusin
14 5 -7728 6928          jsh357
15 1 -9907 6843             WanderingWinder

I ended up with this, might not be the tool of choice as aligning everything is more or less trial and error based, but I like it. Plus I changed the ID's position in the 2nd one which I find even better as players will mostly identify themselves by names.

Code: [Select]
Rank   ID   Rating   Uncertainty                  Name
   1    6    12381          6700                 Watno
   2   10     6244          7299                   ftl
   3   13     3750          7425            lespeutere
   4    4     2420          7256                Tables
   5    3      534          7157         Twistedarcher
   6   15        0          7500      A Drowned Kernel
   7   14        0          7500               jonts26
   8   12        0          7500                 Qvist
   9    8        0          7276               Schneau
  10    7        0          7500                Kirian
  11    2        0          7500             michaeljb
  12    9    -2929          6987                 qmech
  13   11    -5108          7110              markusin
  14    5    -7728          6928                jsh357
  15    1    -9907          6843       WanderingWinder

Code: [Select]
Rank     Rating   Uncertainty                  Name  ID
   1      12381          6700                 Watno   6
   2       6244          7299                   ftl  10
   3       3750          7425            lespeutere  13
   4       2420          7256                Tables   4
   5        534          7157         Twistedarcher   3
   6          0          7500      A Drowned Kernel  15
   7          0          7500               jonts26  14
   8          0          7500                 Qvist  12
   9          0          7276               Schneau   8
  10          0          7500                Kirian   7
  11          0          7500             michaeljb   2
  12      -2929          6987                 qmech   9
  13      -5108          7110              markusin  11
  14      -7728          6928                jsh357   5
  15      -9907          6843       WanderingWinder   1
Logged

DStu

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2627
  • Respect: +1490
    • View Profile
Re: The WW Rating System
« Reply #49 on: April 23, 2013, 10:11:24 am »
0

You might want to use [ table ]

RankIDRatingUncertaintyName
16123816700Watno
21062447299ftl
31337507425lespeutere
4424207256Tables
535347157Twistedarcher
61507500A Drowned Kernel
71407500jonts26
81207500Qvist
9807276Schneau
10707500Kirian
11207500michaeljb
129-29296987qmech
1311-51087110markusin
145-77286928jsh357
151-99076843WanderingWinder

Code: [Select]
[table]
[tr][td]Rank[/td][td]ID[/td][td]Rating[/td][td]Uncertainty[/td][td]Name [/td][/tr]
[tr][td]1[/td][td]6[/td][td]12381[/td][td]6700[/td][td]Watno [/td][/tr]
[tr][td]2[/td][td]10[/td][td]6244[/td][td]7299[/td][td]ftl [/td][/tr]
[tr][td]3[/td][td]13[/td][td]3750[/td][td]7425[/td][td]lespeutere [/td][/tr]
[tr][td]4[/td][td]4[/td][td]2420[/td][td]7256[/td][td]Tables [/td][/tr]
[tr][td]5[/td][td]3[/td][td]534[/td][td]7157[/td][td]Twistedarcher [/td][/tr]
[tr][td]6[/td][td]15[/td][td]0[/td][td]7500[/td][td]A Drowned Kernel [/td][/tr]
[tr][td]7[/td][td]14[/td][td]0[/td][td]7500[/td][td]jonts26 [/td][/tr]
[tr][td]8[/td][td]12[/td][td]0[/td][td]7500[/td][td]Qvist [/td][/tr]
[tr][td]9[/td][td]8[/td][td]0[/td][td]7276[/td][td]Schneau [/td][/tr]
[tr][td]10[/td][td]7[/td][td]0[/td][td]7500[/td][td]Kirian [/td][/tr]
[tr][td]11[/td][td]2[/td][td]0[/td][td]7500[/td][td]michaeljb [/td][/tr]
[tr][td]12[/td][td]9[/td][td]-2929[/td][td]6987[/td][td]qmech [/td][/tr]
[tr][td]13[/td][td]11[/td][td]-5108[/td][td]7110[/td][td]markusin [/td][/tr]
[tr][td]14[/td][td]5[/td][td]-7728[/td][td]6928[/td][td]jsh357 [/td][/tr]
[tr][td]15[/td][td]1[/td][td]-9907[/td][td]6843[/td][td]WanderingWinder [/td][/tr]
[/table]
« Last Edit: April 23, 2013, 10:13:47 am by DStu »
Logged
Pages: 1 [2] 3 4 5  All
 

Page created in 0.058 seconds with 21 queries.