Dominion Strategy Forum

Please login or register.

Login with username, password and session length
Pages: [1]

Author Topic: The CouncilRoom Simulation is Prediction challenge  (Read 8434 times)

0 Members and 1 Guest are viewing this topic.

rrenaud

  • Administrator
  • *****
  • Offline Offline
  • Posts: 991
  • Uncivilized Barbarian of Statistics
  • Respect: +1197
    • View Profile
    • CouncilRoom
The CouncilRoom Simulation is Prediction challenge
« on: June 24, 2011, 03:45:08 pm »
0

rspeer has a dominion bot.  I have been playing with doing winningness prediction.  Geronimo has a dominion simulator.  I have a bunch of so far not all that interesting game result pages.

Why not give users an easy way to see who is going to win the game and also answer the question of which system really understands Dominion the best?

For rspeer's system and my own system, we just need to feed the game state to our predictors and see what they think.  For Gerinomoo's system (and other simulators), we need to be able give it a mid-game game state, and simulate the outcome many times (perhaps expose strategy choices as url params), and then use probability of winning as the predictor.

Other contestants are welcome to join as well.

To get this working really requires us agreeing on an encoding of the mid game states.  I guess it also requires some work for Gerinomoo to support starting games from the middle and exposing a non GUI interface.
« Last Edit: June 24, 2011, 03:48:58 pm by rrenaud »
Logged

rspeer

  • Witch
  • *****
  • Offline Offline
  • Posts: 469
  • Respect: +877
    • View Profile
Re: The CouncilRoom Simulation is Prediction challenge
« Reply #1 on: June 26, 2011, 06:29:50 am »
0

I'd be in.
Logged

Randal FTW

  • Scout
  • ****
  • Offline Offline
  • Posts: 41
  • Respect: 0
    • View Profile
Re: The CouncilRoom Simulation is Prediction challenge
« Reply #2 on: June 30, 2011, 04:48:04 am »
0

Pretty sure im a huge dog to most you guys (it seems like a lot of this forum is top 100 and im barely top 250) but im in.
« Last Edit: June 30, 2011, 04:50:31 am by Randal FTW »
Logged
With No Power Comes No Responsibility

A_S00

  • Spy
  • ****
  • Offline Offline
  • Posts: 84
  • Respect: +41
    • View Profile
Re: The CouncilRoom Simulation is Prediction challenge
« Reply #3 on: July 13, 2011, 02:25:04 pm »
0

Further idea:  Use "winningness prediction" (whichever prediction strategy turns out to be most successful) as fitness function to help simulators like Geronimoo's decide what to do in those rare cases where there are actually non-trivial decisions other than what to buy (i.e., make whichever choice results in the game state with the highest "winningness").

*edit* golem is already looking to do this, isn't it?
« Last Edit: July 13, 2011, 02:27:32 pm by A_S00 »
Logged

rrenaud

  • Administrator
  • *****
  • Offline Offline
  • Posts: 991
  • Uncivilized Barbarian of Statistics
  • Respect: +1197
    • View Profile
    • CouncilRoom
Re: The CouncilRoom Simulation is Prediction challenge
« Reply #4 on: July 13, 2011, 03:22:33 pm »
0

Yup, Golem is doing making buy suggestions based on a model.

Councilroom is pretty close to having probability of winning per turn graphs on each page (i have some not yet released, I want to improve the models before they go public).  Along with the graph, I might add buy suggestions for each turn.

I'd happily write a nice and small interface to the model that speaks in terms of game states and probabilities if someone wants to reuse it.
Logged

rspeer

  • Witch
  • *****
  • Offline Offline
  • Posts: 469
  • Respect: +877
    • View Profile
Re: The CouncilRoom Simulation is Prediction challenge
« Reply #5 on: July 13, 2011, 05:56:42 pm »
0

I'm getting reasonably confident in Golem's ability to evaluate the "winningness" of a deck and suggest appropriate buys. In most cases. Here are the big things I still need to fix:

- Its endgame is dumb. This makes some sense, because playing the endgame is about expanding out the probabilistic game tree and maximizing your chance of ending with the most VP, not about making your deck look more like an ideal deck. An amusing example is that it appears to have kind of learned the Penultimate Province Rule, but incorrectly: I saw it decide a Duchy was better than a Province when there were two Provinces left in the supply. When it was ahead on points.

- It tends to severely undervalue silver when it has a chapel, so that if it were paired with a reasonable card-playing strategy, it would often have to hold onto its coppers for quite a while to avoid killing the deck. I don't know why it does this.

The remaining flaws are API problems that don't affect the underlying deck-fitness function:

- It doesn't know when piles are Embargoed and there currently isn't a way to tell it.

- It doesn't plan ahead when it should know how many coins it's going to have. If you want Golem to buy the right cards with a 4/3 opening, it's better to ask it in the 3/4 order so it doesn't waste its $4 buy on a $3 card it wants exactly one of, or a Chapel.

- Don't tell it it has 4 or more buys, because it will consider the possibilities exhaustively. A somewhat greedy search would be better -- for example, find the best choice with 2 buys, buy the best single card of those two, and re-evaluate until all the buys are used or until the appropriate buy is none.

Other people might be able to make the progress I'm not making on actually playing turns intelligently if they had access to Golem, so I plan to put it up on a server with a JSON API soon. Would this help?

My guess is that's more helpful than expecting other people to actually run the code, as it depends on some rather experimental software, but if people want to get involved in the development of Golem itself I could also work on explaining how to get its dependencies.
« Last Edit: July 13, 2011, 06:01:30 pm by rspeer »
Logged

rrenaud

  • Administrator
  • *****
  • Offline Offline
  • Posts: 991
  • Uncivilized Barbarian of Statistics
  • Respect: +1197
    • View Profile
    • CouncilRoom
Re: The CouncilRoom Simulation is Prediction challenge
« Reply #6 on: July 13, 2011, 06:21:06 pm »
0

As long as the dependencies aren't super obscene and painful, I'd at least be willing to try to get golem working for councilroom.  I'll happily invest an hour to it.

But a json API would be definitely be nice.

I'd like to send a list of game states, and then get back

A) a list of probability win at each state
B) a recommended buy set for that turn
C) maybe a ranked list of alternative buy sets with their 'discount', how much model things the next best options cost in terms of difference in probability to win.
Logged
Pages: [1]
 

Page created in 0.065 seconds with 21 queries.