Dominion Strategy Forum

Please login or register.

Login with username, password and session length
Pages: [1]

Author Topic: Card Leaderboard  (Read 2181 times)

0 Members and 1 Guest are viewing this topic.

markus

  • Apprentice
  • *****
  • Offline Offline
  • Posts: 291
  • Shuffle iT Username: markus
  • Respect: +434
    • View Profile
Card Leaderboard
« on: March 24, 2020, 02:55:03 am »
+7

In the absence of a Qvist card ranking this year, over at the Discord we had the idea for a different approach.

Vote
A small website for submitting pairwise card comparisons is here: https://sites.google.com/view/domrating
There is an optional field for your (user) name. You can always not answer a comparison and it will just not enter the results. You can also call cards similar and this will enter as a tie result. I'll leave it up to you to decide, whether to call it a tie or make one card better than the other. (For Hexes, the idea is to vote for the ones that hurt the most.)

Results
Leaderboards can be calculcated the same way as the leaderboard of players, where cards either win or tie against each other. The results are regularly updated and can be found here:
https://docs.google.com/spreadsheets/d/1CaVOd1pgAgmjJHXPM1tVMVnlJOLDZaq8BxjW4I1NI1E/edit#gid=0

Everybody can submit as much or as little as they want. (There could eventually be alternative leaderboard calculations using weighted or selected responses only.)

I used all the Qvist categories, making Ways a new one. This rating approach allows for pooling all cost categories. On the spreadsheet with the leaderboard, there is a column for the Qvist cost that you can use to filter to find the subset of $3-costs etc. ("temporary filter", you can also download the sheet)
Kingdom cards and Landscapes are not mixed (so far), but Events and Projects are pooled.

The idea is to have it up for some time to get an initial rating. Afterwards, I could leave it open for further submissions or (temporarily) shut it down - not sure what the best approach is. After some time and/or when new cards get added, there could be a boost to cards' uncertainty and a new communal effort to update ratings (similar to the yearly edition of Qvist rating). The advantage is that we could then start from the existing ratings to have closer match-ups from the start as well as better initial ratings.

Interpreting the results
The rating that each card gets follows similar Glicko rules as the Dominion Online leaderboard (rating = mu there).
This means that the difference of ratings between cards, indicates how often it is expected to win in a (hypothetical) match-up. If everyone agreed on the exact order of the cards, you'd expect to see big gaps between each card. Groups of cards that are closer together indicate the opinions vary or "similar" is used.
The formula is approximately: win_probability = 1/(1+exp(-(rating1-rating2)))
There are 3 columns that calculate this win probability of a card compared to the best, the median, and the worst card in the list.

phi is the uncertainty of the rating. It goes down with more submissions and it shows whether the rating difference between two cards is (statistically) significant. (You could add about +/- 2*phi to the rating to get a 95% confidence interval.)
Logged

faust

  • Cartographer
  • *****
  • Offline Offline
  • Posts: 3377
  • Shuffle iT Username: faust
  • Respect: +5142
    • View Profile
Re: Card Leaderboard
« Reply #1 on: March 24, 2020, 03:22:12 am »
0

Thansk for setting up!

I feel like Ways should be ranked together with Landmarks, they fulfill a similar role in changing the game's rules.
Logged
You say the ocean's rising, like I give a shit
You say the whole world's ending, honey it already did

silverspawn

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5301
  • Shuffle iT Username: sty.silver
  • Respect: +3190
    • View Profile
Re: Card Leaderboard
« Reply #2 on: March 24, 2020, 03:59:53 am »
0

When I look at these lists, I always have the following thought process: "this is wrong" -> "I can capitalize on opponents thinking this is the correct ranking!" -> "{thinks about real games}" -> "actually I can't because people don't play as if they believe this" -> "?! :("

I'd be very interested in a list of how often people actually buy stuff in online games, and how that list compares to power ratings. didn't we used to have sth like that?

markus

  • Apprentice
  • *****
  • Offline Offline
  • Posts: 291
  • Shuffle iT Username: markus
  • Respect: +434
    • View Profile
Re: Card Leaderboard
« Reply #3 on: March 24, 2020, 04:16:13 am »
+1

I'd be very interested in a list of how often people actually buy stuff in online games, and how that list compares to power ratings. didn't we used to have sth like that?

There is my google sheet with those statistisc: https://docs.google.com/spreadsheets/d/1M2L7hcY3sbA33OwuZhgPYJWVlMFgJYBdK8cnkbJHmbo/edit#gid=0

you can look at the gains etc of the better player or the winner as well. And there are the summary pictures linked from there. Or collectively at: https://onedrive.live.com/?authkey=%21AJzkg625QzgBbtY&v=photos&id=4375584A8C199C03%215347&cid=4375584A8C199C03
Logged

silverspawn

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5301
  • Shuffle iT Username: sty.silver
  • Respect: +3190
    • View Profile
Re: Card Leaderboard
« Reply #4 on: March 24, 2020, 05:45:01 am »
0

Thanks! Those are super interesting.

markus

  • Apprentice
  • *****
  • Offline Offline
  • Posts: 291
  • Shuffle iT Username: markus
  • Respect: +434
    • View Profile
Re: Card Leaderboard
« Reply #5 on: March 24, 2020, 09:15:26 am »
0

I've added a landscape vs cards comparison on the rating site: https://sites.google.com/view/domrating/landscapes-vs-cards
Whoever likes comparing those classes, can do that now as well. Incoming results will make those two leaderboards a joint one - even though for now I will display them separately. This means, that if the average Landscape is rated higher than the average card, the ratings of all Landscapes will go up and those of the Cards will go down. The difference between 2 cards or 2 landscapes shouldn't be affected, if everything was consistent.

I feel like Ways should be ranked together with Landmarks, they fulfill a similar role in changing the game's rules.
I could do something similar to what I just did with Landscapes vs Cards, if there is enough demand. A clear majority on Discord was against mixing Ways with Projects+Events, but I can see that they do fit better with Landmarks. In general, my guidance were the Qvist ratings and there I would suspect that Ways become a new category. (Having Events and Kingdom Cards of the same cost together in the Qvist rating motivated me to allow for that here as well.)
Logged
Pages: [1]
 

Page created in 0.04 seconds with 21 queries.