Dominion Strategy Forum

Please login or register.

Login with username, password and session length
Pages: [1] 2  All

Author Topic: Dominion Data Mining: Cards that correlate with skill  (Read 15453 times)

0 Members and 1 Guest are viewing this topic.

ben_king

  • Golem
  • ****
  • Offline Offline
  • Posts: 190
  • Shuffle iT Username: ben.king
  • formerly grsbmd
  • Respect: +612
    • View Profile
Dominion Data Mining: Cards that correlate with skill
« on: January 08, 2015, 12:52:17 pm »
+41

Working off suggestions from the previous thread, I've taken the 90,000 game database of games by top-100 players and looked at how the player's skill (represented by their TrueSkill rating) correlates with what cards they gain.  Fortunately, top-100 players also play plenty of low-ranked players, so there are lots of examples of both low- and high-ranked players in the database.

The table below shows two different rankings.  Both measure the correlation coefficient between gaining a certain card and the skill of the player.  Positive numbers mean that the card tends to be bought more often by high-ranked players than low-ranked players.  Negative numbers mean that the card tends to be bought more often by low-ranked players.  The ranking on the left is an unweighted ranking, which means that the correlation is between skill and whether the card gets gained or not (gaining it multiple times doesn't make any difference.  The ranking on the right is a weighted ranking, which means that here the number of times the card is gained makes a difference.    So on the right side, you can say, for example, that good players seem to know to get lots of wharves, whereas bad players seem to overbuy some good cards like Tournament or Bishop.

Unweighted ranking_____________________Weighted ranking
RankCardCorrelationRankCardCorrelation
1Butcher0.1271Governor0.206
2JackOfAllTrades0.1062Wharf0.172
3Wishing Well0.1043JackOfAllTrades0.148
4Vineyard0.0874Hunting Party0.144
5Governor0.0855Wishing Well0.141
6Chancellor0.0816Vineyard0.138
7Duke0.0807Stonemason0.112
8Courtyard0.0778King's Court0.111
9Warehouse0.0719Apothecary0.111
10Masterpiece0.07010Butcher0.107
11Scavenger0.06511Duke0.106
12Fairgrounds0.06212Stables0.100
13Oracle0.05613Scrying Pool0.097
14Journeyman0.05614Horn of Plenty0.096
15Apothecary0.05515Fairgrounds0.095
16Masquerade0.05516Menagerie0.095
17Counterfeit0.05417Oracle0.094
18Duchess0.05118Watchtower0.093
19Stonemason0.05019Masquerade0.088
20Ambassador0.04120Warehouse0.087
......
187Moneylender-0.094187Spy-0.089
188Trade Route-0.096188Remodel-0.093
189Expand-0.096189Forge-0.096
190Treasure Map-0.099190Soothsayer-0.102
191Coppersmith-0.103191Tournament-0.102
192Marauder-0.106192Island-0.105
193Soothsayer-0.107193Expand-0.105
194Talisman-0.113194Baker-0.106
195Transmute-0.115195Feast-0.106
196Spy-0.118196Trade Route-0.107
197Golem-0.118197Transmute-0.109
198Feast-0.123198Tribute-0.114
199Mine-0.128199Bishop-0.119
200Alchemist-0.132200Treasure Map-0.127
201Tribute-0.155201Mine-0.130
202Bishop-0.158202Scout-0.145
203Scout-0.171203Marauder-0.146
204Taxman-0.177204Saboteur-0.168
205Saboteur-0.195205Taxman-0.172
206Pirate Ship-0.262206Pirate Ship-0.204
Logged

jsh357

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2577
  • Shuffle iT Username: jsh357
  • Respect: +4340
    • View Profile
    • JSH Gaming: Original games
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #1 on: January 08, 2015, 01:29:27 pm »
0

This is probably the most interesting data to me so far.  The bottom 5-7 on the left chart says everything I'm always thinking when I see players go for bad plans, though I am somewhat surprised Marauder is down there.  I always thought I ignored it too often.

And of course, Vineyard being near the top on the right makes sense to me.  When I play lower ranked players, I very often go for Vineyard all on my own and laugh as I have 8 9-point cards.
« Last Edit: January 08, 2015, 01:30:47 pm by jsh357 »
Logged
Join the Dominion community Discord channel! Chat in text and voice; enter dumb tournaments; spy on top players!

https://discord.gg/2rDpJ4N

liopoil

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2587
  • Respect: +2479
    • View Profile
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #2 on: January 08, 2015, 01:50:57 pm »
+3

A few of these cards (governor, wharf, hunting party) may be inflated on the right because when both players go for them, more often then not the better player will win the split.
Logged

Awaclus

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 11808
  • Shuffle iT Username: Awaclus
  • (´。• ω •。`)
  • Respect: +12846
    • View Profile
    • Birds of Necama
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #3 on: January 08, 2015, 02:23:40 pm »
+2

A few of these cards (governor, wharf, hunting party) may be inflated on the right because when both players go for them, more often then not the better player will win the split.

Does that really make the results inaccurate, though? If the card is so good that both players usually go for it, it probably deserves to be inflated.
Logged
Bomb, Cannon, and many of the Gunpowder cards can strongly effect gameplay, particularly in a destructive way

The YouTube channel where I make musicDownload my band's Creative Commons albums for free

liopoil

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2587
  • Respect: +2479
    • View Profile
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #4 on: January 08, 2015, 02:26:00 pm »
0

A few of these cards (governor, wharf, hunting party) may be inflated on the right because when both players go for them, more often then not the better player will win the split.

Does that really make the results inaccurate, though? If the card is so good that both players usually go for it, it probably deserves to be inflated.
No, this is measuring cards that are underrated/overrated by lower ranked players, not how good the card is. Fishing Village is a great card, but everyone knows that so it's not on the list. If both players go for it that game should be a wash, but because one player is just better at winning splits, it isn't.
Logged

Awaclus

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 11808
  • Shuffle iT Username: Awaclus
  • (´。• ω •。`)
  • Respect: +12846
    • View Profile
    • Birds of Necama
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #5 on: January 08, 2015, 02:54:26 pm »
0

Fishing Village is a great card, but everyone knows that so it's not on the list.

It would be on the right list if it was a card that you need 654165541 copies of, because if everyone knew that, everyone would always try to get as many as possible, and the better player would always win the split.
Logged
Bomb, Cannon, and many of the Gunpowder cards can strongly effect gameplay, particularly in a destructive way

The YouTube channel where I make musicDownload my band's Creative Commons albums for free

2.71828.....

  • Saboteur
  • *****
  • Offline Offline
  • Posts: 1290
  • Shuffle iT Username: irrationalE
  • Respect: +1322
    • View Profile
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #6 on: January 08, 2015, 03:10:04 pm »
+2

I recognize a full list would take too much space as a post, but could you put it in a file to download or something?
Logged
Man. I had four strips of bacon yesterday. Was one automatically undercooked, one automatically overcooked? No, let's put a stop to that right here, all four strips were excellent.

liopoil

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2587
  • Respect: +2479
    • View Profile
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #7 on: January 08, 2015, 05:46:30 pm »
+2

Fishing Village is a great card, but everyone knows that so it's not on the list.

It would be on the right list if it was a card that you need 654165541 copies of, because if everyone knew that, everyone would always try to get as many as possible, and the better player would always win the split.
Better players don't put more priority in getting Governors, Hunting Parties or Wharves in most games; Better players are better at getting Governors, Hunting Parties, or Wharves. Weaker players open Taxman and then never hit $5, which is why taxman is on the list.
Logged

Merudo

  • Ambassador
  • ***
  • Offline Offline
  • Posts: 32
  • Respect: +29
    • View Profile
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #8 on: January 08, 2015, 08:32:18 pm »
+2

grsbmd, I don't think the correlation coefficient is at all adequate for the "unweighted ranking".

The main problem is that you are trying to estimate an association between two binary variables (high-ranked / low-ranked with buy/don't buy).

EDIT: I thought the player ranks had been dichotomized, but they were not. My apologies.
« Last Edit: January 09, 2015, 01:52:30 am by Merudo »
Logged

ben_king

  • Golem
  • ****
  • Offline Offline
  • Posts: 190
  • Shuffle iT Username: ben.king
  • formerly grsbmd
  • Respect: +612
    • View Profile
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #9 on: January 09, 2015, 12:13:42 am »
0

The main problem is that you are trying to estimate an association between two binary variables (high-ranked / low-ranked with buy/don't buy).
...

Thanks for the good thought Merudo.  Fortunately, I think the problem is not quite that dire.  Player skill is actually a continuous variable, which makes the correlation much more reliable.  That being said, the skill distribution in my dataset is biased by my collection method.  Just so that everyone is aware of what kind of biases might be playing into these results, I've attached a histogram of the skill levels represented.

Logged

WanderingWinder

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5275
  • ...doesn't really matter to me
  • Respect: +4381
    • View Profile
    • WanderingWinder YouTube Page
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #10 on: January 09, 2015, 09:26:50 am »
0

So, the biggest thing to note here is that all of the numbers are tiny. You're not finding anything significant. Well, maybe statistically significant, but not practically so.

ben_king

  • Golem
  • ****
  • Offline Offline
  • Posts: 190
  • Shuffle iT Username: ben.king
  • formerly grsbmd
  • Respect: +612
    • View Profile
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #11 on: January 09, 2015, 11:02:37 am »
+4

So, the biggest thing to note here is that all of the numbers are tiny. You're not finding anything significant. Well, maybe statistically significant, but not practically so.

I'd argue that finding correlations this large is actually fairly substantial.  If we assume that how often you buy a card is independent of other cards (which is a fairly reasonable assumption as far as independence assumptions go, since in full random the chance of getting any two specific cards in a kingdom is ~0.002%), then the r^2 values range from 0.7% to 4%.  This means that statistically, I can explain 4% of the variation in skill among players simply by looking at how often the player buys Governor.  If you sum up the top 20 cards on the weighted list, that explains 29% of the variance in the skill.

That's huge.  This doesn't even include things like how cards are played once they're bought, when to start greening, etc.  So the fact that we can explain so much of the variance in skill simply by a how often a few cards are bought is a really big deal.
Logged

flies

  • Minion
  • *****
  • Offline Offline
  • Posts: 629
  • Shuffle iT Username: flies
  • Statistical mechanics of hard rods on a 1D lattice
  • Respect: +348
    • View Profile
    • ask the atheists
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #12 on: January 09, 2015, 03:47:31 pm »
0

(does anyone know why people talk about explaining X% of the variance and not the deviation? why is [unit^2] more meaningful than [unit]?)
Logged
Gotta be efficient when most of your hand coordination is spent trying to apply mascara to your beard.
flies Dominionates on youtube

flies

  • Minion
  • *****
  • Offline Offline
  • Posts: 629
  • Shuffle iT Username: flies
  • Statistical mechanics of hard rods on a 1D lattice
  • Respect: +348
    • View Profile
    • ask the atheists
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #13 on: January 09, 2015, 03:51:17 pm »
0

is it possible to represent these data as some kind of function like, difference in likelihood of purchase/number purchased as a functino of difference in rank?  These correlations seem hard to interpret.
Logged
Gotta be efficient when most of your hand coordination is spent trying to apply mascara to your beard.
flies Dominionates on youtube

ben_king

  • Golem
  • ****
  • Offline Offline
  • Posts: 190
  • Shuffle iT Username: ben.king
  • formerly grsbmd
  • Respect: +612
    • View Profile
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #14 on: January 09, 2015, 04:53:50 pm »
+1

(does anyone know why people talk about explaining X% of the variance and not the deviation? why is [unit^2] more meaningful than [unit]?)

Probably the main reason that we talk about variance rather than deviation is that variance is always positive (since it's a squared value), whereas deviation can be positive or negative.  So you'd have to use the absolute value of the deviation, which gets messy mathematically.  There's also a nice linear relationship with sum of squared deviations that isn't there when you try to sum absolute deviations (but that's more on the technical side).

is it possible to represent these data as some kind of function like, difference in likelihood of purchase/number purchased as a functino of difference in rank?  These correlations seem hard to interpret.

To get such a function, I would need to run linear regression on this data and pull out those coefficients.  The correlation coefficient (what's presented in the first post) simply measures how close to linear the relationship between skill and gaining a certain card is.  It ignores the strength of that relationship.
Logged

popsofctown

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5477
  • Respect: +2860
    • View Profile
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #15 on: January 09, 2015, 07:53:24 pm »
+4

I have a suggestion for another similar analysis.  (this one is definitely cool)

If only one player in a game buys a card, and the other player does not, what % of the time is the player who bought the card the one with a high skill rating? (You could weight it by the skill gap, but I don't think that would be that helpful)


The way I understand what you did, if both the good and bad player frequently buy a card, it waters down the card's positive score.  Since both players are usually at least good enough to know Ambassador is a good idea it has a weaker score, while Vineyard is much higher up because there's a lot more people out there that haven't advanced enough to know that Vineyards dominants a heck of a lot of boards.  The statistics given are very interesting in a way that is neither better than worse but I'm interested in the result of my suggestion if it interests you.

If I totally misunderstood and that's already what you did, oops.
Logged

c4master

  • Moneylender
  • ****
  • Offline Offline
  • Posts: 167
  • Respect: +56
    • View Profile
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #16 on: January 10, 2015, 07:41:27 am »
+1

Can someone estimate the likelyhood of a card getting +/-0.100 correlation just by chance? Or do we need further assumptions? We have more than 200 cards and I'm wondering whether some outliers could be explained by this.

I'm very surprised to see Moneylender on this list. Seriously, when do you ignore it? When there's better trashing, I guess. Or, when you're going for BM or a slog, maybe. I thought, it would be pretty easy to see when Moneylender is good and when it's not.
Logged

theblankman

  • Witch
  • *****
  • Offline Offline
  • Posts: 461
  • Respect: +383
    • View Profile
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #17 on: January 10, 2015, 10:45:44 am »
0

Working off suggestions from the previous thread, I've taken the 90,000 game database of games by top-100 players and looked at how the player's skill (represented by their TrueSkill rating) correlates with what cards they gain. 
I must've missed the previous thread... has someone put up that whole DB to be downloaded?  I have an experiment or two I'd like to try myself. 
Logged
it's a shame that full-random is the de facto standard

noughtpointzero

  • Pearl Diver
  • **
  • Offline Offline
  • Posts: 14
  • Respect: +20
    • View Profile
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #18 on: January 10, 2015, 01:22:17 pm »
0

Can anyone explain why wishing well is so high?
Logged

A Drowned Kernel

  • 2015 World Champion
  • *
  • Offline Offline
  • Posts: 1067
  • They/Them
  • Respect: +1980
    • View Profile
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #19 on: January 10, 2015, 01:35:31 pm »
+1

Can anyone explain why wishing well is so high?

It's a good card that lower-level players frequently underestimate. Good deck tracking can make the draw quite reliable.
Logged
The perfect engine
But it will never go off
Three piles are empty

Mic Qsenoch

  • 2015 DS Champion
  • *
  • Offline Offline
  • Posts: 1709
  • Respect: +4329
    • View Profile
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #20 on: January 10, 2015, 01:38:42 pm »
+5

Can anyone explain why wishing well is so high?

Because it's awesome.

It's a good substitute for Silver in the opening. It gives you a good shot at hitting $5 after the first shuffle, but then doesn't get in the way of drawing your deck like a Silver will. And of course throughout the game you can still expect to get the Lab effect occasionally and sometimes more than occasionally if you're tracking your deck. And there are some decent combos as well: Apothecary, Wandering Minstrel, Cartographer.
Logged

Jack Rudd

  • Saboteur
  • *****
  • Offline Offline
  • Posts: 1323
  • Shuffle iT Username: Jack Rudd
  • Respect: +1379
    • View Profile
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #21 on: January 10, 2015, 02:27:42 pm »
+2

Can anyone explain why wishing well is so high?
I explain it thus. Or rather, theory explains it thus.
Logged
Centuries later, archaeologists discover the remains of your ancient civilization.

Evidence of thriving towns, Pottery, roads, and a centralized government amaze the startled scientists.

Finally, they come upon a stone tablet, which contains but one mysterious phrase!

'ISOTROPIC WILL RETURN!'

popsofctown

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5477
  • Respect: +2860
    • View Profile
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #22 on: January 10, 2015, 04:14:15 pm »
+6

The only surprise is that Wishing Well is not even higher.  You have

1: Wishing Well is more engine friendly than Silver, and high skill players are more engine leaning than low skill players in general.
2: High skill players are more likely to recognize the good things about Wishing Well and know that it is good on the board, just like most of the other stuff that did well on this list

And the big bazooley
3: Since good players deck track better than bad players, there's a nontrivial amount of the time that it is correct play for the good player to buy it AND for the bad player not to buy it.  There is an actual difference in how the card is going to perform for each of the players.  That's true of few other cards. 
Logged

DG

  • Governor
  • *****
  • Offline Offline
  • Posts: 4074
  • Respect: +2624
    • View Profile
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #23 on: January 10, 2015, 04:30:04 pm »
0

I played Rabid last week and we'd emptied the wishing well pile before his 5th turn, which is quite good going. Essentially in any deck where silver is a poor card a wishing well is likely to be a strong card. Top players will buy and gain them in preference to silver (at the right times).

The Council Room stats for isotropic included win rates for when players gained/bought (or didn't buy/gain) a kingdom card. These might be interesting figures to compare again now we have all the cards on Goko.
« Last Edit: January 10, 2015, 04:33:28 pm by DG »
Logged

HiveMindEmulator

  • Mountebank
  • *****
  • Offline Offline
  • Posts: 2222
  • Respect: +2118
    • View Profile
Re: Dominion Data Mining: Cards that correlate with skill
« Reply #24 on: January 10, 2015, 07:55:09 pm »
+9

Can anyone explain why wishing well is so high?

Because it's Stef's avatar.
Logged
Pages: [1] 2  All
 

Page created in 0.123 seconds with 21 queries.