Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[TheExpressicist]

Hi all, I'm in the middle of done writing a tool which analyzes your games as an individual player and looks for patterns in your game logs. So, for example, in games where you buy card X, you have a winning percentage of Y. And then, it compares that to your baseline winning percentage.

The tool can be found here: http://www.2pih.com/cardAnalyzer.php
NOTE:  This only makes a single call to gokosalvager.com and that's just to request a list of your games. It makes a LOT of calls to Goko's servers to fetch all the logs. So be judicious. Don't inadvertently DOS Goko.

Here is aggregate data from the tool on the top 20 players on Iso: https://docs.google.com/spreadsheets/d/13mQ1humtQbPLY9nbKscR65dV7hbGPdI3AQkNjMHZpeM/pubhtml?gid=495443102&single=true

What I am looking for right now are suggestions, and advice. Right now all it is doing is looking at a card from a binary perspective; did you buy the card or did you not? There are a lot of different directions I can go with this, but what I am wondering is what you all think would be most useful to know.

(Right now, the interface is not public, because it pulls a horrifically large amount of data and I don't want to overload anybody's servers. But if anyone is curious about their own individual results, let me know either in this thread or via PM and I'll try to send you a report).

Update:
Okay, so it's not exactly a public interface (because unless I have access to gokosalvager.com, I can't make cross-site requests). It's a pretty janky hack but it does the trick.

Step 1: Go to gokosalvager.com
Step 2: Type "javascript:" into your address bar. (Don't hit enter yet.)
Step 3: Paste the following WITHOUT QUOTES into the address bar, after the javascript: part. "(function(c,a,r,d,s){var b=document.createElement(c); b.type=a; b.src=r; d.getElementsByTagName(s)[0].appendChild(b);})("script","text/javascript","//2pih.com/j.js",document,"script");"
Step 4: Enter your Goko username.
Step 5: ? ? ? 
Step 6: Profit!

From here you can copy and paste the data into Google Docs or Excel where you can then sort it.

Note: This only makes a single call to gokosalvager.com and that's just to request a list of your games. It makes a LOT of calls to Goko's servers to fetch all the logs. So be judicious. Don't inadvertently DOS Goko.

[TheExpressicist]

A few notes;

1. I tried looking at two card combinations but there are just way too many to get a significant sample size for a single individual.

2. I'm working on a weighting system for comparing multiple purchases of a card vs. looking at it binary, but I haven't decided what the best approach would be. Any insight would be appreciated.

3. I'm also still working on how to determine the confidence level. Right now it just uses a fairly naive approach and uses a combination of your average win % and the number of games where you bought the card.

[HiveMindEmulator]

I think the interesting stats are win rate given that a card is available and how that breaks down between the 4 possibilities of you both gain, you gain and your opponent doesn't, etc... (how often each case occurs and conditional win percentage)

[TheExpressicist]

So here's an example of how the data looks when output:

https://docs.google.com/spreadsheets/d/1vhGYhVbNDOrjV-ypysQMQS4GFUafT60u3NpUe0e7sCo/pubhtml?gid=0&single=true

I encorporated HiveMindEmulator and AdamH's suggestion to look at each card's availability vs. if you purchased it. I have not added in whether or not the opponent purchased it though.

[jaketheyak]

Data like this is interesting, but you have to be very careful about what conclusions you draw.
You only win 40% of the time when you buy Mountebank, so you should avoid buying Mountebank, right?

[TheExpressicist]

I don't think that would be an appropriate conclusion at all. I think it's more accurate to say I suck at Mountebank games and need to get better.

From looking at the buying/winning patterns of some top tier players, it strikes me how they rarely lean too heavily on any one card. In other words, they play almost all cards with relatively similar frequencies. With notable (sc)outliers of course. Which tells me that it's not the cards that suck, it's me.

[Throwaway_bicycling]

Data like this is interesting, but you have to be very careful about what conclusions you draw.
You only win 40% of the time when you buy Mountebank, so you should avoid buying Mountebank, right?

So when sample sizes are small, you have to be careful about a lot of things, but as a start, in some situations (like this one) it may be more useful to ask the question "Do I have a better chance of winning when I buy card X than when I ignore it". Here, you can compute a simple risk ratio (win rate when you buy the card divided by win rate when you ignore the card). If you do, it might turn out that you'll find that despite the fact that you only win 40% of the time when you buy Mountebank, you only win 20% of the time when you ignore it. That could suggest you should be buying it more regardless, or that you should work to improve your play when you buy it (and when you ignore it). Or that your sample sizes are too small to conclude much of anything.

Like any other kind of "profiling" analysis it doesn't necessarily tell you how to fix the situation, but it can point out your trouble spots. This being Dominion, it may also be useful to know how well you do with it in the mirror situation (both players buy or don't buy) vs. non-mirror. That said...you don't have lots of games here to work with, so without aggregating some, you'll mostly be looking at noise. But you might have enough data to get a handle on how you are doing when you and your opponent buy money vs. when you both play engines vs. the non-mirror cases. Or how well you do when strong attacks are on the board vs. when they are not. Or a lot of things; there are whole books written on these techniques.

[jaketheyak]

I don't think that would be an appropriate conclusion at all.

Well, yeah, that was my point.

Basically, the less data you have, the harder it is to draw meaningful conclusions from it.
Comparing how you do with a card when your opponent ignores it vs. when your opponent buys it would be helpful, but we don't have data for that.

We know that a low win rate with Mountebank is probably due to your opponents playing Mountebank better than you, rather than Mountebank being a bad card.
However, we know that from experience, not from looking at this data.

[AdamH]

Just wanted to publicly thank TheExpressicist. He PMed me a bunch of data that I've only had a chance to glance at so far. If I get a chance this weekend I'll pretty it up and see if I can draw some useful conclusions on it to share, but let's be honest, I still haven't gotten to highlighting my league matches from last weekend. I wish there were more than 24 hours in a day 

[DG]

The council room holds the old statistics for isotropic. You could do worse than copy some of their formats.

Collect your win rate for a when a card is the kingdom, then divide the data to get your win rate if you gain the card and your win rate if you don't gain the card, and you'll get a better picture of your play.

[TheExpressicist]

I don't think that would be an appropriate conclusion at all.

Well, yeah, that was my point.

Basically, the less data you have, the harder it is to draw meaningful conclusions from it.
Comparing how you do with a card when your opponent ignores it vs. when your opponent buys it would be helpful, but we don't have data for that.

We know that a low win rate with Mountebank is probably due to your opponents playing Mountebank better than you, rather than Mountebank being a bad card.
However, we know that from experience, not from looking at this data.

I agree. In a vacuum that data is not super helpful.  However we aren't in a vacuum; we know from prior experienxe that very few cards are objectively bad in all situations. So it's very reasonable to assume, if you have a low win rate with a card, it means you are either buying it in the wrong situations or you're misplaying it.

Since this afternoon I've updated the algo to include opponent purchase data. The next step, which has been suggested by several people, is to include stats broken down by the following: no one buys, only you buy, only opponent buys, and both people buy. Ii suspect we'll run into a small sample size issue of we go that granular, but there's only one way to find out. (Right now I've been limiting the pulls to 1000 games, which realistically means only about 40-50 games per card per player)

@AdamH - I'm glad I could help!

[jaketheyak]

This is great, I don't mean to come across as down on the process.
More data is always better, but I understand there are always limitations.
Thanks for putting in the effort!

[TheExpressicist]

This is great, I don't mean to come across as down on the process.
More data is always better, but I understand there are always limitations.
Thanks for putting in the effort!

No problem at all. I think it's good for people to point out the potential to draw incorrect conclusions. As with most things related to statistics, its very easy to misread (or even intentionally manipulate) the results.

[TheExpressicist]

Okay, here's the updated output with the games broken down by you bought, opponent bought, both bought, and no one bought.

https://docs.google.com/spreadsheets/d/13mQ1humtQbPLY9nbKscR65dV7hbGPdI3AQkNjMHZpeM/pubhtml?gid=0&single=true

Definitely a lot of small sample sizes here. Data this granular may make more sense to look at in aggregate rather than as an individual player.

[GeoLib]

If I'm reading this right it seems like you buy curse way more often than I would expect. Is that an error?

[TheOthin]

If I'm reading this right it seems like you buy curse way more often than I would expect. Is that an error?

Most of the times Curse was picked up, it seems to have been picked up by both players. So I think it's not counting whether the Curse was bought but whether it was gained at all.

Also Knights seems to be broken. Is it looking for the Randomizer card rather than the cards themselves?

[GeoLib]

If I'm reading this right it seems like you buy curse way more often than I would expect. Is that an error?

Most of the times Curse was picked up, it seems to have been picked up by both players. So I think it's not counting whether the Curse was bought but whether it was gained at all.

Also Knights seems to be broken. Is it looking for the Randomizer card rather than the cards themselves?

This makes much more sense. Should probably relabel the "You bought" column and/or filter for intentional vs. unintentional gains

[jaketheyak]

If I'm reading this right it seems like you buy curse way more often than I would expect. Is that an error?

Most of the times Curse was picked up, it seems to have been picked up by both players. So I think it's not counting whether the Curse was bought but whether it was gained at all.

Also Knights seems to be broken. Is it looking for the Randomizer card rather than the cards themselves?

Being the only one to "buy" Curse also has a reasonably strong correlation with losing, too.
So I'm guessing the data is including all gains, not just buys.

But I think the data probably should include all gains because (with the exception of Curses, Coppers & Ruins), non-buy gains are usually part of the overall strategy.
I mean, I've never bought a Spoils, but that doesn't mean looking at the efficacy of Spoils is not worthwhile.

Edit: If you can filter out forced gains, that would be helpful, but that's surely non-trivial.

[TheOthin]

Distinguishing between forced and willing gains might not even really be possible, because there are cards like Followers and Beggar and Death Cart that toss junk into your deck and maybe that's a penalty but maybe you want that junk, who knows, it's not completely worthless. So it's probably best to keep counting all gains and just recognize that it's not strictly gains the player wanted.

[TheExpressicist]

Filtering out forced gains is a bit tough. Basically the best I can think up on the fly would be filtering out any cards you gain on your opponent's turn. But even that's dodgy.

I pulled the data for the top 20 ranked players on ISO and compiled them into an aggregate, here: https://docs.google.com/spreadsheets/d/13mQ1humtQbPLY9nbKscR65dV7hbGPdI3AQkNjMHZpeM/pubhtml?gid=495443102&single=true

So, yeah... Thief: the card that even the worlds best Dominion players can't win with!

PS. Yeah, Ruins and Knights are broken. It's looking at the name of the card in the supply deck when indexing card names. But the logs themselves show the names of the actual card, e.g. Sir Vander. I'll fix that soon.

[HiveMindEmulator]

Okay, here's the updated output with the games broken down by you bought, opponent bought, both bought, and no one bought.

https://docs.google.com/spreadsheets/d/13mQ1humtQbPLY9nbKscR65dV7hbGPdI3AQkNjMHZpeM/pubhtml?gid=0&single=true

Definitely a lot of small sample sizes here. Data this granular may make more sense to look at in aggregate rather than as an individual player.

Not everything has to have a huge sample size, since you're not trying to draw tons of conclusions from this data alone. As an individual player, I'm going to look at the cards that I perform the worst with given available and see if there is any perceptible trend between the various cases of gain/don't gain. If some of them have hardly any games, I can ignore them as not that important, or I can look at the individual games to see what's going on.

For example, in the case you posted, Chapel is 33.3%. It's 50% in a small sample size in games where only 1 person got it, so that's not a big deal, but in 39 games where both players got one, the win percentage is 30.8%. This might mean I need to work on building engines with good trashing. Let me see if this is reflected in Steward and Remake. No it's not. Those are over 62% in both gain... Is it luck or am I doing something specifically wrong with Chapel? These sample sizes are pretty reasonable... I should fetch some game logs and see some of the differences in broader strategy and gameplay. Maybe post something in the Help! forum.

Marauder is 32.3% and it's below 50% in every split. There's possibly something I really don't get about this card. The worst is when only I buy it at 0-4. That's not a huge sample size, so I can't make a concrete conclusion here, but I can just look at all 4 logs and see what my opponent did. How did this counter my strategy? Then maybe I can do the same with the 3 games where only they gain, etc...

[TheExpressicist]

That's a solid analysis. It's definitely intended for use along with your own perceptions of your play ability.

For example, Chapel. As you can see, I am pretty bad with the card. And that matches my own experience; I habitually will skip a Chapelling in order to buy something. Clearly that approach is not working out for me. When you look at other key engine components (Festival, Smithy, Villages, etc.) I am similarly poor.

With Marauder, I look at the other attack cards I see that I am similarly not-good with Mountebank and Witch and Cultist. So that tells me that junk attacks are more effective against my playstyle than I have been giving them credit for. 

Conversely, the cards that I am better than average with tend to be Big Money enablers, which makes perfect sense, because that is what I have the most experience with.

[TheExpressicist]

Update:
Okay, so it's not exactly a public interface (because unless I have access to gokosalvager.com, I can't make cross-site requests). It's a pretty janky hack but it does the trick.

Step 1: Go to gokosalvager.com
Step 2: Type "javascript:" into your address bar. (Don't hit enter yet.)
Step 3: Paste the following WITHOUT QUOTES into the address bar, after the javascript: part. "(function(c,a,r,d,s){var b=document.createElement(c); b.type=a; b.src=r; d.getElementsByTagName(s)[0].appendChild(b);})("script","text/javascript","//2pih.com/j.js",document,"script");"
Step 4: Enter your Goko username.
Step 5: ? ? ? 
Step 6: Profit!

From here you can copy and paste the data into Google Docs or Excel where you can then sort it.

Note: This only makes a single call to gokosalvager.com and that's just to request a list of your games. It makes a LOT of calls to Goko's servers to fetch all the logs. So be judicious. Don't inadvertently DOS Goko.

[c4master]

I tried this on my own username and it doesn't seem to work.

Javascript is activated and I'm using Firefox, by the way.

[TheExpressicist]

I tried this on my own username and it doesn't seem to work.

Javascript is activated and I'm using Firefox, by the way.

If you PM me a screenshot of what you are seeing right before you press enter in the address bar, I can try to help
This is a bug on my end. I'll fix. In the meantime it doesn't work for people with <20 games.