Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[WanderingWinder]

CR stats in terms of overall card effectiveness are pretty meaningless.

I couldn't disagree more.  While CR stats are not perfect, it's not like any other evidence we have at our disposal (primarily simulation and individual games) is perfect either.  CR stats are usually in the ballpark of "correct" and if you're careful with them (i.e. understanding WHY Silver has such a high "Win Rate Without") they're basically always useful.

Well, I suppose I should say next to meaningless, rather than straight meaningless. But just because the other options have big flaws doesn't mean that this is better. I have no idea what you mean by 'CR stats are usually in the ballpark of "correct"'. I mean, they're absolutely correct, as far as I know. They don't lie. It's just, they only tell you what they tell you, and you really can't read all that much out of them. 'People buy card X all the time' does NOT mean that it's a good card. It means it's a popular card. "People win all the time when they get diadem" does NOT mean it's exceptionally good, or that it's the third-best prize. Correlation does not imply causality here. If anything, the winrates of how people do when they get cards tell me more about how the card is overrated or underrated than how good the card is. Now, in large bulk, you can get very vague generalizations out of the stats here, from the community's winrates as a whole. The top cards TEND to be better than the bottom cards. BUT this doesn't mean that it's true in any particular case. I mean, even in your post, you're doing this. You say that you have to understand WHY silver has a great win rate without. It's because among isotropic-ers, the people who ignore silver entirely TEND to be building a very successful engine. But how do we know that? Not from the stats. We know that because we already know it. Like, loan has a low winrate without. Does this mean that we should be preferring loan to silver all the time? The stats suggest that. But it's not such a hot idea. You have to analyze things apart from the stats to get anything useful. And in this case, you can get basically all that usefulness WITHOUT the stats. So they're not that useful really at all.

But the bigger thing is that all the time you quote your own "effect with(out)" stats as evidence that a card is good or bad. But that is TOTALLY wrong. What those tell us is that YOU are better/worse with/without the card, against the people you play against, in comparison to the 'average' isotropic player, against the people they play against. Which is to say, it tells us jack squat about how good the card is in general, let alone on a specific board. So you have great tournament numbers. Fine. But that does not tell us that tournament is a good card. It tells us that you're good with it, maybe. Definitely nothing more than that. The same thing, but to a lesser extent, to your own personal win with/without and given availability stats (or mine, or anyone else's); it just doesn't tell you anything about the card objectively.

[chwhite]

But the bigger thing is that all the time you quote your own "effect with(out)" stats as evidence that a card is good or bad. But that is TOTALLY wrong. What those tell us is that YOU are better/worse with/without the card, against the people you play against, in comparison to the 'average' isotropic player, against the people they play against. Which is to say, it tells us jack squat about how good the card is in general, let alone on a specific board. So you have great tournament numbers. Fine. But that does not tell us that tournament is a good card. It tells us that you're good with it, maybe. Definitely nothing more than that. The same thing, but to a lesser extent, to your own personal win with/without and given availability stats (or mine, or anyone else's); it just doesn't tell you anything about the card objectively.

It's not wrong per se, though sure it is incomplete, as it is one data point among many.  If most high-level players have good effects with Tournament, then that would certainly be evidence that tournament is a high skill card, and if most such players had negative effects, then I'd take that as evidence that it's a high luck card.  You're right that just one person is not enough to make sweeping generalizations, but if we could filter Popular Buys by level, then we'd get a much better and more useful picture.

To that end... let's pan out just a little wider.  Here are the current top 10 players on Isotropic, with effects with and without for two cards.  The first one is Tournament, a strong card that most people consider to be unacceptably high-luck but which I've been defending.  The other card is Mountebank, a similarly strong card which I would expect to be much more high-luck.  What can we see?

Code: [Select]

Player Tourn E/W Tourn E/WO Mount E/W Mount E/WO

WanderingWinder -0.79 2.62 0.36 0.80
-Stef- 1.83 N/A (!) -0.40 0.33
Marin -1.59 N/A (!) -1.50 32.56(!!!!!)
Obi Wan Bonogi -3.23 0.96 -3.17 1.39
jonts26 -0.69 0.68 -0.67 2.21
Fabian 0.25 1.29 -0.69 2.39
chwhite 1.15 0.68 -2.40 0.36
AVeryHappyFish 0.81 -0.96 -1.15 1.11
ARTjoMS 0.01 -1.85 -0.42 1.31
RomaNorgy -1.68 1.07 -2.08 0.69
What I found was a lot more small sample size oddities than expected.  Stef and Marin appear to have bought Tournament every single game, and Marin appears to be 1-0 in games where he skipped the Mountebank, leading to that ridiculous Effect Without.   And those are just the most egregious things.

Smaller oddities crop up when you look closer: Obi Wan Bonogi has a horrible Effect With for Tournament, so that's a possible data point for it being a luck-heavy card- except that Obi has far fewer Tournament games than anything which isn't Governor, Possession, or Hinterlands, leading me to the conclusion that he's probably vetoed it a lot and maybe hasn't bothered to develop Tournament strategy to the extent he has for other cards.  Also of interest: somehow AVeryHappyFish has only bought Mountebank 69.6% of the time, still with a positive Effect Without.  That's some impressive avoidance.

But anyway, what I'm seeing is a lot more consistent for Mountebank than Tournament.  All the top players have a negative Effect With for Mountebank except for you, and everyone does better when they avoid it.  The "effect without" unanimity can be explained, of course, by the fact that since we all know Mountebank is such a strong card, we only avoid it when there's a good alternative/counter, which is not the case for Isotropic at large.  But the fact that most top players have negative Effects With is, I think, evidence for the proposition that it's a card which is, in some way or another, anti-skill.

Tournament is more all over the map.  There's a tendency towards negative Effects With and positive Effects Without, but it's not so nearly pronounced as Mountebank.  Looking at these numbers (in conjunction with the real-game experience of myself and others, of course), I'm willing to say that Tournament is probably more luck-intensive than the average card, but it's far from the worst offender.  And also that Stef and Marin should try skipping it someday.

[werothegreat]

Finally - some bad cards that are random, but generally bad so its hard to have rage.
Saboteur
Tribute
Thief
Pirate Ship

You take back what you said about Pirate Ship!  I've won using Pirate Ships several times.

[WanderingWinder]

I intend to respond to chwhite's stuff in a bit, both on the stats front and also on the tournament front. But I want to jump in now, while I don't have much time, and say that I don't think tribute is particularly good. Just not nearly in the class of bad of the other cards Captain Frisk mentioned.

[Axxle]

Smaller oddities crop up when you look closer: Obi Wan Bonogi has a horrible Effect With for Tournament, so that's a possible data point for it being a luck-heavy card- except that Obi has far fewer Tournament games than anything which isn't Governor, Possession, or Hinterlands, leading me to the conclusion that he's probably vetoed it a lot and maybe hasn't bothered to develop Tournament strategy to the extent he has for other cards.

Isn't that because he plays a lot of Colony games, which make Tournament less effective (meaning he isn't as comfortable with it when he does buy it)

[O]

Tribute is not bad. If you aren't running a tight ship on actions it's quite good in an engine, and is very very good in BM with light trashing.

[rrenaud]

Here is a nice empirical definition of luck driven cards.

Cards which are the most luck driven are the ones that make player based winner prediction the most difficult.

[WanderingWinder]

This is directed mainly at chwhite, but also at everyone.
First off, on the stats: You talk about having small sample size effects. You do realize that the 'I have only skipped it in 5 games' isn't the only small sample size issue here, right? Like, those 10 players have what, 50k games? Or something? That's actually not a huge sample to start with, but then you switch it over to adding in they have to have yea-so-many with the card in question... it becomes sketchy. But then the bigger thing is that I have, personally, over 10.5 thousand Iso games. Those come against so many different opponents on so many different boards at some many different points in my skill level, from first and from second position, and they all just get lumped together. So but the biggest thing here is opposition. I have tons and tons of games against level 20- players. And I have tons and tons of games against 40+ players. And almost certainly, my mix of cards that I get in against these different opponents is pretty different. The issue here is that absolutely every one of these stats is based off of winrate. So first of all, my winrate is way more dependant on the strength of my opposition than most anything else around, including the specific cards present. So I have pretty bad effects with Hinterlands cards. Why? I'll tell you - mostly, it's because these are the newest cards, and it's most recently that I've been playing thousands of games against the other strongest of the strong. So even if I may be actually playing better now, just looking at winrate, it looks like I'm playing worse. Because I win less. But that's not on the cards I get, or the skill I'm showing, but mostly 'cause my opponents are better now. It's the same reason my level keeps going up as my winrate keeps going down. Now, what does that mean? Well it means there's pretty big incongruities and biases in your data, which is going to largely invalidate it. And yes, over large sample size, this should even out, BUT... I think we're well, well off from that. On top of the effect with(out) numbers being totally skewed if you don't account for how often the things are bought, and also that your opponents very often buy the same cards you do when you buy them, which skews the numbers again.

Now tournament. Tournament, I have long held, is a card that takes a lot of skill to play, but which also is one of the swingier cards in the game. It's not the swingiest (which I think is swindler, even above treasure map), but it's up there. Now why do I hate tournament? Do my tournaments get blocked and/or do my opponent's? This is the same issue, almost exactly that mountebank has. Mountebank itself is pretty swingy, and this subsumes most of its luck just right there, which isn't even close to the biggest thing about it. Ok. Now, I agree with you, you can really build a deck around getting your tournaments to go off, and basically this means an engine. But before we even get to the main issues there, let's talk about when there's no viable engine. So here, you very often just skip the tourney if there's a good BM option out there. But your opponent can get lucky and hit (a la treasure map) and while you're not lost, it puts you in a very difficult situation. But okay, let's say you're building that awesome engine. Now your opponent can still spike an early 8 for a province, or play a more BMish thing, hit early, or just block your tournaments which really slow your build-up, and again, you're backfooted, even though you're playing right. But okay, finally, and this is the really big thing. I know that I need to play an engine, you know that engine is the way to go, we both do it, invariably one of us gets to the key prize first and... unlike with most engines, that's basically just it, because I have no recourse to make this advantage back up later. And unlike most cases where I have no recourse, the game is going to go on for a good while longer while I get pummeled, because it's not that you've beaten me to the mega-turn. And there's LOTS of luck (lots of P1 too, but lots and lots of luck) as to who hits first. You probably don't want to wait to get the province until you can reliably draw your whole deck to buy province, because that'll be too late. So maybe you're drawing half your deck, or probably a little more. Okay, that means there's like a 30% chance or something that you hit in any given turn (you have to draw the tourney with the province before you play the tourney, and you have to worry about other tourneys getting blocked... eh, it's going to be higher than this sometimes, but probably not more than 60% or so, at least very early on), and the same for your opponent. Okay... that's pretty swingy as to who gets it first. So I mean, this is way better than if you've built it poorly, and you have, like a 10% chance on any given shuffle, let along turn. But that's still a lot of luck. And it's just so very frustrating to get your door punded in by followers, a curser AND discarder, or to get your usually-a-peddler turned into nothing. So that's why it's among the handful of my least favourite cards. And I don't know about the other top players in general, though I do know Marin has often told me he absolutely hates it.

[popsofctown]

So I thought about the councilroom problem and I think I have an answer of what you could shoot for if you wanted to build a filter.

For every two players X and Y, where X has won most of the games he played against Y (I think councilroom doesn't store ranks from certain points in time so this is the best reference point), look at the games where Y "upset" X.  The cards contained in that game are more likely to be variant cards.  It's a really weak correlation but I think that should work with sufficiently massive available information.


I'd like to say, niche cards that some players understand at high or low levels that do not correspond closely to the level at which they understand the game in general, such niche cards  will appear and mirror variant cards.  If the skill with those cards is sufficiently "arbitrary" that it behaves like a random variable.  For thought experiment, suppose that every game involving Cutpurse, the players played a game of Starcraft at the beginning, and whoever wins the game has his Cutpurse miss the reshuffle.  I happen to have played Starcraft before, so I would have a better chance of beating ehunt if Cutpurse is on the board.  Cutpurse is just one card so his inability to play Starcraft is not a reflection on his skill as a whole, it's not even wrong of him to neglect his study of Cutpurse/Starcraft because it's unlikely to appear at any point in time.
And of course, I also have a better chance of beating ehunt if a truly variant card is on the board, because I'm going to make poorer decisions than he.

Anyway, I think such a Councilroom filter would find both variant cards and niche cards.  I'm not sure how to filter out the niche cards, but I think we can make some guesses at what they are (Possession is both variant and niche, Tournament is both variant and niche, Swindler is pure variant, Inn is likely niche, but not variant.)

[blueblimp]

For every two players X and Y, where X has won most of the games he played against Y (I think councilroom doesn't store ranks from certain points in time so this is the best reference point), look at the games where Y "upset" X.  The cards contained in that game are more likely to be variant cards.  It's a really weak correlation but I think that should work with sufficiently massive available information.

I actually did exactly this a while ago: http://forum.dominionstrategy.com/index.php?topic=1521.msg28157#msg28157. The results were a mixed bag. Sometimes they looked meaningful, sometimes not.

[popsofctown]

I think we should be open to the data challenging our expectations.  I think the data filtering is totally legitimate. 

Remember what I said about niche cards?  Duke is probably there as a niche.  Adventurer represents people who understand Dominion as a whole.

I don't know what to think of smithy, I haven't played a board weak enough where I needed to buy it in quite sometime.  It might have ended up filtering down to only pure BM/Smithy games.

[popsofctown]

Pearl Diver and other weak cards scored high too.  That's probably just measuring that less skill is required to play 8 card kingdoms etc.

[WanderingWinder]

I think we should be open to the data challenging our expectations.  I think the data filtering is totally legitimate. 

Remember what I said about niche cards?  Duke is probably there as a niche.  Adventurer represents people who understand Dominion as a whole.

I don't know what to think of smithy, I haven't played a board weak enough where I needed to buy it in quite sometime.  It might have ended up filtering down to only pure BM/Smithy games.

I think we should be open to the data challenging our expectations.  I think the data filtering is totally legitimate. 

Remember what I said about niche cards?  Duke is probably there as a niche.  Adventurer represents people who understand Dominion as a whole.

I don't know what to think of smithy, I haven't played a board weak enough where I needed to buy it in quite sometime.  It might have ended up filtering down to only pure BM/Smithy games.

So... the stats only tell us stuff if we filter out lots of stuff based on what we already know -> basically the stats tell us nothing. But it makes us sound like we have evidence when we pick and choose the stuff that seems to agree with us.

[popsofctown]

No.  I'm arguing that Duke is ranked first, which creates a meaningful biconditional, "Duke is a variant card or Duke is a specialized card".  I conjecture that it's a specialized card, it could be variant.  It has to be one of the two, though.

[popsofctown]

Possession is neither variant nor specialized, so accusations that Possession is variant are false.

[rrenaud]

This is from sample of 2.5 M iso games. 

code is here:

https://github.com/rrenaud/dominionstats/commit/600047addfc61a9aa1ee7d53cc454ca01eef60d8

data is here: http://councilroom.com/card_luck2.txt.bz2 format is reasonably simple to figure out,

name1 : level1 : name2 : level2 : win_prob_for_winner : supply1,supply2,...

The measure is how hard a time trueskill had at predicting the winner, measured in conditional entropy given a card was in the supply. 

http://en.wikipedia.org/wiki/Conditional_entropy

Goons 0.689 ± 0.002
Colony 0.694 ± 0.001
Platinum 0.694 ± 0.001
Bishop 0.695 ± 0.002
Ambassador 0.695 ± 0.002
Grand Market 0.696 ± 0.002
Black Market 0.696 ± 0.002
Worker's Village 0.697 ± 0.002
Chapel 0.697 ± 0.002
Peddler 0.698 ± 0.002
Young Witch 0.698 ± 0.003
Expand 0.699 ± 0.002
Masquerade 0.699 ± 0.002
Hunting Party 0.699 ± 0.002
Witch 0.700 ± 0.002
Ghost Ship 0.700 ± 0.002
Hamlet 0.700 ± 0.002
Mountebank 0.700 ± 0.002
Minion 0.700 ± 0.002
King's Court 0.700 ± 0.002
Sea Hag 0.700 ± 0.002
City 0.700 ± 0.002
Scrying Pool 0.700 ± 0.002
Menagerie 0.700 ± 0.002
Festival 0.701 ± 0.002
Torturer 0.701 ± 0.002
Steward 0.701 ± 0.002
Apprentice 0.701 ± 0.002
Familiar 0.701 ± 0.003
Forge 0.701 ± 0.002
Walled Village 0.702 ± 0.002
Trade Route 0.702 ± 0.002
Fishing Village 0.702 ± 0.002
Tournament 0.702 ± 0.002
Village 0.703 ± 0.002
Quarry 0.703 ± 0.002
Market 0.703 ± 0.002
Mint 0.703 ± 0.002
Ill-Gotten Gains 0.703 ± 0.003
University 0.703 ± 0.002
Pirate Ship 0.703 ± 0.002
Ironworks 0.703 ± 0.002
Upgrade 0.703 ± 0.002
Talisman 0.703 ± 0.002
Bridge 0.704 ± 0.002
Great Hall 0.704 ± 0.002
Vineyard 0.704 ± 0.002
Outpost 0.704 ± 0.002
Scheme 0.704 ± 0.003
Remake 0.704 ± 0.002
Border Village 0.704 ± 0.003
Remodel 0.704 ± 0.002
Warehouse 0.704 ± 0.002
Thief 0.704 ± 0.002
Salvager 0.704 ± 0.002
Wharf 0.705 ± 0.002
Spice Merchant 0.705 ± 0.003
Haven 0.705 ± 0.002
Mining Village 0.705 ± 0.002
Farming Village 0.705 ± 0.002
Rabble 0.705 ± 0.002
Contraband 0.705 ± 0.002
Potion 0.706 ± 0.001
Workshop 0.706 ± 0.002
Bazaar 0.706 ± 0.002
Fairgrounds 0.706 ± 0.002
Woodcutter 0.706 ± 0.002
Native Village 0.706 ± 0.002
Alchemist 0.706 ± 0.002
Watchtower 0.706 ± 0.002
Horse Traders 0.707 ± 0.002
Throne Room 0.707 ± 0.002
Fortune Teller 0.707 ± 0.002
Governor 0.707 ± 0.003
Lookout 0.707 ± 0.002
Mine 0.707 ± 0.002
Archivist 0.707 ± 0.038
Spy 0.707 ± 0.002
Apothecary 0.708 ± 0.002
Farmland 0.708 ± 0.003
Monument 0.708 ± 0.002
Tactician 0.708 ± 0.002
Smugglers 0.708 ± 0.002
Loan 0.708 ± 0.002
Transmute 0.708 ± 0.002
Envoy 0.708 ± 0.002
Inn 0.708 ± 0.003
Venture 0.708 ± 0.002
Crossroads 0.708 ± 0.003
Militia 0.708 ± 0.002
Nomad Camp 0.708 ± 0.003
Trading Post 0.708 ± 0.002
Duke 0.708 ± 0.002
Wishing Well 0.708 ± 0.002
Bureaucrat 0.708 ± 0.002
Secret Chamber 0.708 ± 0.002
Bank 0.708 ± 0.002
Herbalist 0.708 ± 0.002
Pawn 0.708 ± 0.002
Horn of Plenty 0.708 ± 0.002
Royal Seal 0.708 ± 0.002
Nobles 0.708 ± 0.002
Cartographer 0.709 ± 0.003
Mandarin 0.709 ± 0.003
Counting House 0.709 ± 0.002
Vault 0.709 ± 0.002
Harvest 0.709 ± 0.002
Highway 0.709 ± 0.003
Island 0.709 ± 0.002
Explorer 0.709 ± 0.002
Navigator 0.709 ± 0.002
Saboteur 0.709 ± 0.003
Cellar 0.709 ± 0.002
Coppersmith 0.709 ± 0.002
Shanty Town 0.709 ± 0.002
Pearl Diver 0.709 ± 0.002
Conspirator 0.709 ± 0.002
Cutpurse 0.709 ± 0.002
Feast 0.709 ± 0.002
Moneylender 0.709 ± 0.002
Laboratory 0.710 ± 0.002
Margrave 0.710 ± 0.003
Courtyard 0.710 ± 0.002
Develop 0.710 ± 0.003
Jester 0.710 ± 0.002
Hoard 0.710 ± 0.002
Philosopher's Stone 0.710 ± 0.002
Tunnel 0.710 ± 0.003
Haggler 0.710 ± 0.003
Tribute 0.710 ± 0.002
Baron 0.710 ± 0.002
Harem 0.710 ± 0.002
Fool's Gold 0.710 ± 0.003
Gardens 0.710 ± 0.002
Possession 0.710 ± 0.003
Adventurer 0.710 ± 0.002
Silk Road 0.710 ± 0.003
Oracle 0.710 ± 0.003
Scout 0.710 ± 0.002
Stash 0.711 ± 0.002
Oasis 0.711 ± 0.003
Golem 0.711 ± 0.002
Council Room 0.711 ± 0.002
Stables 0.711 ± 0.003
Moat 0.711 ± 0.002
Noble Brigand 0.711 ± 0.003
Swindler 0.711 ± 0.002
Embargo 0.712 ± 0.002
Caravan 0.712 ± 0.002
Chancellor 0.712 ± 0.002
Treasury 0.713 ± 0.002
Duchess 0.713 ± 0.003
Treasure Map 0.713 ± 0.002
Library 0.713 ± 0.002
Smithy 0.713 ± 0.002
Trader 0.713 ± 0.003
Merchant Ship 0.714 ± 0.002
Lighthouse 0.714 ± 0.002
Cache 0.714 ± 0.003
Jack of All Trades 0.714 ± 0.003
Embassy 0.714 ± 0.003

[cherdano]

Hah, I knew why I didn't post in this thread! 
I think this conditional entropy measures a combination of two things:
- How much skills on kingdoms involving this card is correlated to skills overall, and
- to what extend skills on kingdoms involving this card affect the outcome.

For example, maybe Merchant Ship or Cache are partly up there because good players don't realize how good it is?

It's interesting that we all missed out on Embassy and JoaT. With Embassy it might be a combination of two things:
- It really simplifies the game if BM+Embassy is the best strategy.
- The luck related to Embassy is a bit more subtle - $P2 instead of $P3 on a familiar board on turn 4 is more dramatic than drawing Embassy on turn 3 or turn 4 - and quite a bit of the large swing only comes late in the game (big difference whether we get to that Embassy on the last reshuffle).
JoaT? Maybe just that the game become too simple?

[DG]

Hinterlands cards are probably high in that ranking list since they are the latest expansion. High ranked players will have lost games while learning the cards and then played comparatively few games afterwards to rebalance the stats.

[rrenaud]

Hinterlands cards are probably high in that ranking list since they are the latest expansion. High ranked players will have lost games while learning the cards and then played comparatively few games afterwards to rebalance the stats.

That's a good point.  Maybe a graph of this vs time would be useful.  I could also try discarding the first X games the players see each card.

[Matt_Arnold]

I am so incredibly glad I posted this question. This is really what I was hoping for. Thank you to you all for these contributions.

[cayvie]

The biggest thing I see in that list is that Goons is far and away the least luck-based card.

[blueblimp]

The biggest thing I see in that list is that Goons is far and away the least luck-based card.

I'm a little surprised. There's a ton of skill in using Goons (and lots of people are bad at it), but there is a luck component in that if you can get a Goons play off before your opponent gets a Goons, it puts you at a pretty big advantage.

[rrenaud]

But if/when to green and how to end the game are pretty important/skill intensive parts of skilled Goons play.

[Matt_Arnold]

Can I ask a favor? Can I get a new version of that list now that Dark Ages has been out for a while? I'm designing sets of cards for a tournament, and would like to include a good number of high-skill cards.

[Watno]

Noone said Minion? Seiously?