Dominion Strategy Forum

Please login or register.

Login with username, password and session length
Pages: [1]

Author Topic: machine learning on Isotropic logs  (Read 3246 times)

0 Members and 1 Guest are viewing this topic.

InfinitePerplexity

  • Herbalist
  • **
  • Offline Offline
  • Posts: 8
  • Respect: 0
    • View Profile
machine learning on Isotropic logs
« on: January 23, 2012, 12:25:13 am »
0

Hi all, I've been teaching myself how to do machine learning in Python lately, using the NumPy package and log data downloaded from Isotropic.  My goal is to write a "kingdom analyzer" that looks at a ten-card kingdom and gives advice that would help a beginning or intermediate player understand what sort of deck the collective wisdom of skilled Isotropic players would build for that kingdom.

I've come up with a sort of "minimum viable implementation" and tested it on five days of game logs.  Currently it does a good job picking two-and-three card combos and counters (e.g. buy Ironworks if Garden is available, but not if Bishop is also available), and it could probably find larger combos if I downloaded more data.  But it's poor at designing a coherent deck, and it still makes "wtf" recommendations pretty regularly (I suspect the model I'm using overfits severely.)  Of course, I'm just a level 18, so maybe the model is right and I'm wrong :)

Anyway, I'm curious to see if anyone else has done experiments along these lines, or would like to share thoughts and strategies.  My current model puts the classifiers themselves into wrappers, so it would be pretty easy to swap out algorithms.  I may put the code up on GitHub, although (1) I code like a scientist, so parts of my code are pretty bad and (2) I've never put anything up on GitHub before so I'm not really familiar with how it works. 
Logged

Qvist

  • Mountebank
  • *****
  • Offline Offline
  • Posts: 2400
  • Shuffle iT Username: Qvist
  • Respect: +4085
    • View Profile
Re: machine learning on Isotropic logs
« Reply #1 on: January 23, 2012, 03:20:02 am »
0

No, I've never done anything like that, though I really like to see it in practise how this works. It sounds really interesting.
So, maybe you can share it anywhere (GitHub, etc) or give us some sample outputs of your program.

rspeer

  • Witch
  • *****
  • Offline Offline
  • Posts: 469
  • Respect: +877
    • View Profile
Re: machine learning on Isotropic logs
« Reply #2 on: January 24, 2012, 08:45:37 am »
0

rrenaud and I have both tried this using his dominionstats codebase (the same code that runs CouncilRoom). It's on GitHub at:

https://github.com/rrenaud/dominionstats

In particular, https://github.com/rrenaud/dominionstats/blob/master/game_state_features.py contains a lot of functions for extracting features from game states and writing them in forms that various machine learning tools understand.

Neither of us have had very much success, but there are a lot of things to try, and you might do a lot better.
Logged

InfinitePerplexity

  • Herbalist
  • **
  • Offline Offline
  • Posts: 8
  • Respect: 0
    • View Profile
Re: machine learning on Isotropic logs
« Reply #3 on: January 25, 2012, 01:12:20 am »
0

The model does two kinds of output, once you've trained it.  One displays something about the learned buy rules for a specific card; for the ad hoc decision tree classifier I wrote, it gives something like this:

>>> print mymodel["Ironworks"]
* Gardens:
  * Bishop:
    - No Buy (16%)
  * No Bishop:
    * Monument:
      - No Buy (16%)
    * No Monument:
      * Young Witch:
        - No Buy (28%)
      * No Young Witch:
(snipped for readability, goes on to basically say you usually buy Ironworks if Gardens is around)
* No Gardens:
  * Great Hall:
    * Trade Route:
        - Buy (60%)
    * No Trade Route:
(snipped for readability)

The other thing it does is it takes a list of kingdom cards, converts it to a boolean input vector, runs the appropriate ten classifiers to produce an output vector, which it then converts back to something human readable:

>>> mymodel.classify("Highway, Nomad Camp, Harvest, Cartographer, Golem, Talisman, Vineyard, Island, Haggler, and University.")

'Cartographer', 'Island', 'Haggler', 'Talisman', 'Nomad Camp', 'Highway'
{'Golem': 0.083333333333333329, 'Cartographer': 0.90000000000000002, 'Island': 0.72727272727272729, 'University': 0.14285714285714285, 'Haggler': 0.66666666666666663, 'Vineyard': 0.4375, 'Talisman': 0.5625, 'Nomad Camp': 0.97222222222222221, 'Harvest': 0.023809523809523808, 'Highway': 0.69999999999999996}

...which is a mix of good and bad advice, to say the least.  My gut says the approach of predicting each card separately is fundamentally flawed, and I have some ideas for how to improve on it, but I haven't had the chance to try them out.

@rspeer, what sort of input and output did you guys try?
Logged

MasterAir

  • Alchemist
  • ***
  • Offline Offline
  • Posts: 39
  • Respect: +8
    • View Profile
Re: machine learning on Isotropic logs
« Reply #4 on: January 25, 2012, 11:26:14 am »
0

I think it's a fun project, it's hard, but you probably already knew that.

One thing that might be interesting is to see if your machine learning technique can make the "is there a viable engine in this setup" decision effectively.  One could try to correlate the number of actions played per turn with whether or not a player won in a particular kingdom. 

One of the toughest things with a dominion learning bot is the phase space it has to work in.  Despite the large number of games played on Isotropic, there can only be a handful of games played with any particular kingdom set.  You may have to give your machine learner some assistance grouping cards into categories.  The problem you have then is that the information you get from the bot is highly dependent on the info you give it.
Logged

blueblimp

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2849
  • Respect: +1559
    • View Profile
Re: machine learning on Isotropic logs
« Reply #5 on: February 16, 2012, 08:18:17 pm »
0

I just did something a little like this. My goal was to get some idea of which cards are most skill-intensive, in the sense that when they are in the supply, you see fewer upsets (lesser-skilled player defeating more-skilled player, according to the leaderboard). I don't know much about machine learning though so I just used a least-squares solver.

I don't know if anyone else is interested, but this seems like an appropriate place to post it. =)

Example card ranking, from running on days 20120203 through 20120214, including only players level 25 and up, from most-skill-requiring to least-skill-requiring:
Code: [Select]
0.08516417: Adventurer
0.08456600: Possession
0.08354641: Harem
0.08184241: King's Court
0.08152772: City
0.08026821: Cartographer
0.07940645: Treasure Map
0.07905757: Cutpurse
0.07900007: Torturer
0.07820899: Mountebank
0.07797046: Warehouse
0.07794976: Margrave
0.07706310: Council Room
0.07590907: Apothecary
0.07456282: Trader
0.07446936: Goons
0.07336511: Remodel
0.07328238: Horn of Plenty
0.07298849: Library
0.07270732: Gardens
0.07264993: Tournament
0.07206545: Steward
0.07094222: Border Village
0.07059525: Workshop
0.07033349: Fishing Village
0.06985302: Treasury
0.06904109: Ironworks
0.06859853: Contraband
0.06834661: Nobles
0.06716353: Upgrade
0.06709585: Black Market
0.06592823: Ghost Ship
0.06592215: Ambassador
0.06532851: Farmland
0.06502409: Stash
0.06407454: Apprentice
0.06401693: Haggler
0.06384017: Alchemist
0.06358155: Explorer
0.06356953: Market
0.06328401: Nomad Camp
0.06294425: Throne Room
0.06191573: Quarry
0.06172529: Embargo
0.06152768: Moat
0.06144618: Expand
0.06101341: Baron
0.06098042: Wishing Well
0.06065459: Mint
0.06033086: Bazaar
0.06028343: Conspirator
0.06026394: Highway
0.05953535: Inn
0.05925004: Monument
0.05912377: Spice Merchant
0.05908763: Hoard
0.05908333: Oracle
0.05900228: Native Village
0.05887850: Masquerade
0.05870625: Fairgrounds
0.05862115: Moneylender
0.05860537: Lookout
0.05859416: Forge
0.05852813: Fortune Teller
0.05803387: Transmute
0.05778421: Cellar
0.05764033: Worker's Village
0.05652846: Woodcutter
0.05633609: Embassy
0.05628023: University
0.05592414: Feast
0.05589341: Royal Seal
0.05589257: Village
0.05501166: Wharf
0.05495642: Vault
0.05474288: Scout
0.05458196: Bishop
0.05440966: Smugglers
0.05438207: Mine
0.05427687: Tribute
0.05421006: Tunnel
0.05412809: Mandarin
0.05400339: Chancellor
0.05379845: Crossroads
0.05379040: Venture
0.05372546: Duchess
0.05351463: Scrying Pool
0.05346366: Mining Village
0.05344670: Island
0.05322741: Develop
0.05320709: Courtyard
0.05317686: Secret Chamber
0.05271292: Menagerie
0.05266542: Talisman
0.05254311: Laboratory
0.05229258: Pirate Ship
0.05211550: Horse Traders
0.05177014: Noble Brigand
0.05084035: Golem
0.05080142: Hamlet
0.05060647: Watchtower
0.05057027: Counting House
0.04996165: Loan
0.04979356: Bureaucrat
0.04975353: Minion
0.04949364: Silk Road
0.04931505: Saboteur
0.04885632: Walled Village
0.04876879: Ill-Gotten Gains
0.04820306: Festival
0.04788748: Grand Market
0.04765027: Vineyard
0.04755092: Salvager
0.04715652: Herbalist
0.04682846: Stables
0.04678177: Haven
0.04624035: Witch
0.04609654: Sea Hag
0.04576599: Jester
0.04534426: Philosopher's Stone
0.04526584: Scheme
0.04513696: Bridge
0.04511840: Pearl Diver
0.04487112: Thief
0.04475541: Swindler
0.04429258: Coppersmith
0.04409065: Spy
0.04401970: Trade Route
0.04340535: Farming Village
0.04331577: Jack of All Trades
0.04309553: Militia
0.04306024: Caravan
0.04293130: Pawn
0.04253476: Great Hall
0.04240333: Harvest
0.04205920: Remake
0.04076288: Trading Post
0.04065376: Governor
0.04053754: Outpost
0.04033976: Envoy
0.03950563: Lighthouse
0.03787837: Navigator
0.03695280: Chapel
0.03551853: Oasis
0.03502646: Shanty Town
0.03480394: Hunting Party
0.03213149: Bank
0.03157986: Tactician
0.03077700: Peddler
0.02922237: Cache
0.02914586: Rabble
0.02834067: Fool's Gold
0.02681132: Merchant Ship
0.02346290: Smithy
0.02067544: Familiar
0.02063283: Duke

Some of these things make sense: a lot of cards that are sometimes-traps appear near the top (Possession, King's Court, City, Treasure Map) and there are some strong high-variance cards that appear near the bottom (Familiar, Fool's Gold, various cards good for big money like Smithy, Envoy, Merchant Ship).

Some of the rankings make no sense at all though--apparently Duke requires the least skill of any card? And Adventurer is the most skill-intensive card of all??? Running this on more data might help. =)
Logged
Pages: [1]
 

Page created in 0.055 seconds with 21 queries.