Dominion Strategy Forum

Please login or register.

Login with username, password and session length
Pages: [1]

Author Topic: The limits of simulation  (Read 5140 times)

0 Members and 1 Guest are viewing this topic.

Quadell

  • Scout
  • ****
  • Offline Offline
  • Posts: 44
  • Respect: +107
    • View Profile
The limits of simulation
« on: April 12, 2012, 10:10:25 am »
+1

Simulation won't exactly duplicate real-world play, obviously, but for some strategies that doesn't seem to matter all that much. Double-Jack is Double-Jack. But there's a very subtle problem I encountered recently that I think demonstrates the limits of simulation when determining the power of a strategy.

I was attempting to see how good a "Workshop Gardener" strategy was against other standard strategies, and I tweaked it in all kinds of ways. I finally found that against standards like Big Money, Village-Torturer, Double-Jack, etc., the best strategy was to buy all the workshops first, and then buy gardens, buying up estates with spare cash as you can along the way.

Code: [Select]
{
  name: 'Fastest Workshop-Gardener'
  author: 'Quadell'
  requires: ["Workshop", "Gardens"]
  gainPriority: (state, my) -> [
    "Workshop"
    "Gardens"
    "Estate"
    "Copper"
  ]
  torturerPriority: (state, my) -> [
    'curse'
  ]
}

This is very fast, and seemed to beat nearly every simulated strategy I could throw at it. It beats Big Money 96% of the time and Double Jack 59% of the time, whereas other Workshop-Gardener strategies (such as the one described in the Gardens article on DS) were slower and had lower percentages. I thought I knew how to win Dominion (at least when Workshop and Gardens are on the board) better than the experts.

But...

A very small number of real games showed me how wrong I was. Although this strategy is the best at beating players who ignore Workshop and Gardens, few decent players will do that (unless they see something better on the board). And when "Fastest Workshop Gardener" goes up against a similar strategy that switches to Gardens earlier, it loses badly. For instance, I call the following strategy "Workshop-Gardener switch after 4", and it more closely simulates what's described in the Gardens article:

Code: [Select]
gainPriority: (state, my) -> [
    "Workshop" if my.countInDeck("Workshop") < 4
    "Gardens"
    "Workshop"
    "Estate"
    "Copper"
  ]

Although it only beats Big Money 83% of the time, and Double Jack a measly 34% of the time, it beats "Fastest Workshop-Gardener" 72% of the time. So is "Workshop-Gardener switch after 4" the best strategy? Not against "Workshop-Gardener switch after 2"! It seems to me that the best Workshop-Gardener strategy depends entirely on what your opponent does. I like to open double Workshop, and if my opponent does as well then I immediately switch to Gardens. But if my opponent starts with only 1 or even 0 Workshops, I keep buying Workshops instead of Gardens until I feel I have to switch. It's almost a game of chicken: it's best to put off Gardens buys, but you really don't want to be the second player to buy one.

And this is for trying to find the best strategy for a very simple artificial situation, where Workshop and Gardens are the only things I buy. In a real game I'll buy a good $2 or an excellent $3 instead of Estates, and there are probably other card interactions that will effect this strategy in subtle ways. There's obviously a lot about this strategy that I'll only learn through real-game experience, no matter how much simulation I do. But even for this artificially constrained situation, it's easy to overestimate the value of simulations in the absence of experience.
Logged

WanderingWinder

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5275
  • ...doesn't really matter to me
  • Respect: +4381
    • View Profile
    • WanderingWinder YouTube Page
Re: The limits of simulation
« Reply #1 on: April 12, 2012, 10:24:42 am »
+1

Garbage in, garbage out

DStu

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2627
  • Respect: +1490
    • View Profile
Re: The limits of simulation
« Reply #2 on: April 12, 2012, 10:29:33 am »
0

In this case of course the solution would be to test if you are the only one going for Workshops, and gain more than 2 Workshops if it is the case, otherwise go for Gardens. In Dominiate this is quite easy, in Geronimoo's sim I think it's more difficult (unless there's something new I've missed).

But in the broad picture of course you are right. You have to react on your opponent, and a 2-card strategy can't do this. It might be possible to do it on a fixed set, but there despite maybe some fun at a competition the results are of limited use.

For example it could be usefull for the DoubleJack to 3-pile you if it is leading and there is just 1 Estate left. But it doesn't, it takes the Provinces/Gold/Duchy, and gives you a chance to catch up (maybe push your 8 Gardens to +1vp). The DoubleJack could deny you some Gardens, but it doesn't (usually that's a bad idea anyway, but who knows...).
You might play against a City, where it might be a good idea not to take the last WS. On the other hand the City-player could just buy it, but a generic City-bot of course doesn't.

So you can tweak 2 strategies against each other and see what might help whom, and this might also help you, but you never really get a strategy that works no matter what you opponent does.

Logged

rrenaud

  • Administrator
  • *****
  • Offline Offline
  • Posts: 991
  • Uncivilized Barbarian of Statistics
  • Respect: +1197
    • View Profile
    • CouncilRoom
Re: The limits of simulation
« Reply #3 on: April 12, 2012, 10:30:35 am »
0

It just means the simulated strategy shouldn't be head down; it should take into account other players decks.  Your post be more appropriately titled as "the limits of simulating 5 line of code strategies."

There is no reason that one bot can't contain a strategy that plays tons of workshops then gardens strategy against a standard, 4+ prov strategy, and few workshops, early gardens against an opponent's workshop strategy.
Logged

Geronimoo

  • Saboteur
  • *****
  • Offline Offline
  • Posts: 1059
  • Respect: +868
    • View Profile
    • Geronimoo's Dominion Simulator
Re: The limits of simulation
« Reply #4 on: April 12, 2012, 10:33:37 am »
0

If you put in more work, it's possible to create strong bots that are able to modify their strategies depending on what the opponent does. A lot can be learnt from this.

Take for instance the Simulation tournament won by WanderingWinder where he tuned an Ironworks/Silk Road rush strategy so it could beat non-rush decks by gaining lots of Ironworks, but change its behaviour completely when playing a mirror match and going for a longer game.
Logged

Quadell

  • Scout
  • ****
  • Offline Offline
  • Posts: 44
  • Respect: +107
    • View Profile
Re: The limits of simulation
« Reply #5 on: April 12, 2012, 10:52:30 am »
0

Your post be more appropriately titled as "the limits of simulating 5 line of code strategies."

Well, if the goal were simply to beat Double Jack, then that 5-line strategy is probably close to optimal. But the goal of beating a likely opponent is done very differently, which is what my post was trying to get at. Perhaps "The limits of simulation alone, without actual playtesting" would express it better?

Quote
There is no reason that one bot can't contain a strategy that plays tons of workshops then gardens strategy against a standard, 4+ prov strategy, and few workshops, early gardens against an opponent's workshop strategy.

It's true, and I've been writing just that. But I never would have realized that was important to do without real-game, non-simulated experience with the cards.

And Geronimoo, wow, Wandering Winder's sim is a thing of beauty. I'll be studying that for a while.
Logged

DStu

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2627
  • Respect: +1490
    • View Profile
Re: The limits of simulation
« Reply #6 on: April 12, 2012, 11:06:28 am »
+1

It's true, and I've been writing just that. But I never would have realized that was important to do without real-game, non-simulated experience with the cards.

It's possible. Take the bot you think is optimal, and try to beat it. Look at it and think about: "What does it want. How can I prevent it from doing that". Test it, if it doesn't work, think more. If it works, go back to the first bot and think how you can nevertheless achieve what you want, despite the first bot distrubing you. In a manner that also works against the "usual" case.

There is a tournament over at http://forum.dominionstrategy.com/index.php?topic=2193.0 , and I think it's maybe mostly about this question. You have an idea of a strong bot. Now you think about what can beat it. First of course stronger bots, so just take them, but relatively fast, you can think of nothing that can beat it in a vacuum. But now you have to think about: How does it do in a mirror? How can tweak it to win the mirror (I think the first tournament was mostly about this, because as the results show it was clear that there is at least one other person with IW/SR). But even here it might be that other people think of at least the same core cards. So you've got some other cards to improve, and defend against the mirror, and defend against other things.
And then still: What might disturb my bot? I rely on getting 10 WishingWells. So how does it perform my opponent also takes WWs? What it it takes my crucial Transmute?
Logged

timchen

  • Minion
  • *****
  • Offline Offline
  • Posts: 704
  • Shuffle iT Username: allfail
  • Respect: +234
    • View Profile
Re: The limits of simulation
« Reply #7 on: April 12, 2012, 03:20:14 pm »
0

For me, I think the simulator is very good at understanding the baseline for some strategies. The biggest limitation is that the simulator has no reasoning. It can tell you how a given algorithm fare with another, but does not tell you why.

For example, the double-jack strategy simulates well. But in practice I am not as impressed. The thing is that drawing the first jack late, as well as having the two jacks colliding hurt a lot. This is something you can't see from the simulation if you don't look carefully enough into the logs.

It is probably entirely fine for other people as long as they get reliable results; that is, the result is stable against fine tuning to some extent. For me, I don't like to play following experiences from a robot. I like to play following heuristics and logic.

So the problem mentioned by OP is indeed a problem I think; if you scale up the bot to handle the player interaction as well as some other considerations, the simulation implies only as much as your initial ansatz. That is, you start with some reasoning to handle specific situations and the simulator can optimize some numbers around it. It however does not tell you which way to start is best. Even worse, the simulator does not tell you how each part of the reasoning you put in interacts with each other. The more advanced and sophisticated the bots are; the less heuristic understanding you can get out from it.

Probably people better at simulation will not agree with my perspective. But the worst of all is that this is a game of chances. One can put all his time into a particular combination (say workshop and gardens) with lots of considerations. But at best he can only improve his chances against me from 50% to 100% in a game with those two cards. And most of the time it is probably only from 50% to 70% given the best tuning. This amounts to one match in hundreds if not a thousand, with the current number of cards. Honestly, I couldn't care less.
Logged

blueblimp

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2849
  • Respect: +1559
    • View Profile
Re: The limits of simulation
« Reply #8 on: April 12, 2012, 03:54:28 pm »
+1

For me, I don't like to play following experiences from a robot. I like to play following heuristics and logic.

In my opinion, the value of simulations is that they can inform the heuristics and logic used to play the game. Without the ability to run many games in a simulation, it can be hard to tell what's happening only due to luck.

(And I agree that jack and especially double-jack are hugely overrated. It's not the fastest BM+X, as it loses by a roughly ~~5-10% margin to Masquerade, Courtyard, and Wharf, for example. It's not immune against attacks either, as it loses against Witch+BM and Mountebank+BM. I don't understand why it seems to have this image of being some crazy unbeatable big money strategy.)
« Last Edit: April 12, 2012, 04:01:43 pm by blueblimp »
Logged

ftl

  • Mountebank
  • *****
  • Offline Offline
  • Posts: 2056
  • Shuffle iT Username: ftl
  • Respect: +1345
    • View Profile
Re: The limits of simulation
« Reply #9 on: April 12, 2012, 04:56:09 pm »
+3

 I think the criticism of the OP sort of misses the point of WHY you would want to simulate something. The simulator isn't an AI; there's no particular expectation for it to play the game optimally in all circumstances and against all opponents.
 
What it is is a tool for answering questions about the game. If I want to know - what's better, strategy X or strategy Y? - how do I find out?

Well, you can sort of think really really hard about it, use your intuition, and come up with an answer. But that doesn't give you any new information, that's just you formalizing what you already think.
You can try logically thinking through all the possibilities, but since Dominion is a game of chance, it's *really* hard to accurately weigh the probabilities of collision and missing shuffles versus the benefits different cards give you.
You can try to play more games to figure it out, but as timchen pointed out, since any individual card combo comes up only in a fraction of games, and even then playing it well versus playing it badly might only give you a 20% margin of error, the effect of good strategy will be extremely difficult to separate from the effect of luck, so you'll never really know.
You can repeatedly play games with it with a friend, but that takes a very dedicated friend, a LOT of time, and is still vulnerable to random chance due to low sample sizes and misremembering bad/good wins or losses.
Or, you could simulate it - write the code for strategies X and Y, and see which one is actually better in those limiting cases where those are the two things you try. And based on the results, make conclusions to apply to your own game. In some cases, that will give you new information that would have been very difficult to get otherwise. (Obviously, that also has caveats - like, it doesn't take into account other possible strategies, and so on. )


Like, the Workshop-Gardens example. Yes, the neither of the two strategies - 9 workshops first, or 2 workshops and then Gardens - is optimal for all situations. But it does give you the general heuristics to use - if your opponent is NOT contesting you for workshops or Gardens, you want to get as many Workshops as you can, whereas if they ARE contesting you for Workshops/Gardens, then switch to Gardens after you get 2 workshops. It lets you think at a higher level in-game - what you need to worry about is "is my opponent contesting me for gardens" rather than the more basic "how many workshops should I get".
Logged

Geronimoo

  • Saboteur
  • *****
  • Offline Offline
  • Posts: 1059
  • Respect: +868
    • View Profile
    • Geronimoo's Dominion Simulator
Re: The limits of simulation
« Reply #10 on: April 12, 2012, 05:54:03 pm »
0

What ftl said!

It's a tool and if used intelligently, you'll learn something out of it despite its limitations.
Logged

theory

  • Administrator
  • *****
  • Offline Offline
  • Posts: 3603
  • Respect: +6121
    • View Profile
    • Dominion Strategy
Re: The limits of simulation
« Reply #11 on: April 12, 2012, 06:11:32 pm »
0

Perhaps Geronimoo is too modest to say it, but I'd say his rapid ascent of the leaderboard is a good illustration of how playing with the simulators is a good way to improve your game :)
Logged
Pages: [1]
 

Page created in 1.746 seconds with 20 queries.