Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Sparafucile]

Hey guys.  I've been working on creating some visualizations for Dominulator, and I came across something I thought you would find interesting.   The tl;dr is that performing analysis by averaging data at a given turn is useful for analyzing openings, it has some drawbacks when analyzing the end game.  For that, a different approach is needed.

Consider the following strategy involving Salvager, lookout, Library, Highway and Festival.   We open with a salvager and lookout, trash quickly, picking up highways festivals and a couple of libraries.   The idea is to get an uber turn where you have lots of coin, lots of buys, and provinces are cheap.  Here's the precise purchase/trash/action order.

Code: [Select]

private static ICardPicker PurchaseOrder()
            {
                var highPriority = new CardPickByPriority(
                     CardAcceptance.For(Cards.Province, ShouldBuyProvinces),
                     CardAcceptance.For(Cards.Duchy, gameState => CountAllOwned(Cards.Province, gameState) >= 3),
                     CardAcceptance.For(Cards.Estate, gameState => CountOfPile(Cards.Province, gameState) <= 1),
                     CardAcceptance.For(Cards.Salvager, gameState => CountAllOwned(Cards.Copper, gameState) >= 6 && CountAllOwned(Cards.Salvager, gameState) == 0),
                     CardAcceptance.For(Cards.Lookout, gameState => CountAllOwned(Cards.Copper, gameState) >= 6 && CountAllOwned(Cards.Lookout, gameState) == 0),
                     CardAcceptance.For(Cards.Silver, gameState => CountAllOwned(Cards.Silver, gameState) + CountAllOwned(Cards.Festival, gameState) < 2)
                     );
               
                var buildOrder = new CardPickByBuildOrder(
                    CardAcceptance.For(Cards.Festival),
                    CardAcceptance.For(Cards.Library),
                    CardAcceptance.For(Cards.Festival),
                    CardAcceptance.For(Cards.Highway),
                    CardAcceptance.For(Cards.Highway),
                    CardAcceptance.For(Cards.Festival),
                    CardAcceptance.For(Cards.Festival),
                    CardAcceptance.For(Cards.Library),
                    CardAcceptance.For(Cards.Festival)
                    );

                var lowPriority = new CardPickByPriority(
                    CardAcceptance.For(Cards.Highway));

                return new CardPickConcatenator(highPriority, buildOrder, lowPriority);
            }

            private static bool ShouldBuyProvinces(GameState gameState)
            {
                return CostOfCard(Cards.Province, gameState) <= 4 || CountAllOwned(Cards.Province, gameState) > 0;
            }

            private static CardPickByPriority ActionOrder()
            {
                return new CardPickByPriority(
                           CardAcceptance.For(Cards.Lookout),
                           CardAcceptance.For(Cards.Highway),
                           CardAcceptance.For(Cards.Festival),
                           CardAcceptance.For(Cards.Salvager),
                           CardAcceptance.For(Cards.Necropolis),
                           CardAcceptance.For(Cards.Library)
                           );
            }           

            private static CardPickByPriority TrashOrder()
            {
                return new CardPickByPriority(                           
                           CardAcceptance.For(Cards.Curse),
                           CardAcceptance.For(Cards.Lookout, gameState => CountAllOwned(Cards.Copper, gameState) <= 4),                                               
                           CardAcceptance.For(Cards.Estate),
                           CardAcceptance.For(Cards.OvergrownEstate),
                           CardAcceptance.For(Cards.Hovel),
                           CardAcceptance.For(Cards.Necropolis),
                           CardAcceptance.For(Cards.Silver, gameState => gameState.Self.ExpectedCoinValueAtEndOfTurn == 4 && CardBeingPlayedIs(Cards.Salvager, gameState)),
                           CardAcceptance.For(Cards.Copper),
                           CardAcceptance.For(Cards.Silver));
            }     
How well, how does this engine compare to big money?  Quite well in fact.   This engine wins 82% of the time:


A more detailed breakdown shows that the engine wins by quite a large margin a majority of the games.  8 points or more ...


And the average coin graph doesn't immediately go against expectations.  Big money is ahead in average spending power, but the engine supercedes eventually.


So it looks like the engine does quite well vs big money.   Lets take a look at the average victory points per turn.

Hmm.  That's weird.  It looks like on average, at any given turn, bigmoney is ahead.   Seems like the engine never gets ahead.  Perhaps the notion of an average is misleading.  Consider for example 3 games being played with the following scores:  30-25, 31-28, 10-40.   Even though the first player wins a majority of the games, the second players average score would be higher.   Let's try something other than average.  How about we just count who is ahead at every turn, and ignore by how much. 


Equally unexpected.  It seems like probabilistically, the engine is never winning - yet it usually wins games.  How is this possible? 

I thought about it for a bit, and I realized that the problem is with averaging data based on turn count.  This is misleading for analyzing the end of the game.  This particular engine is going to peak in strength at around turn 14-16 or so.   It will then attempt to buy many provinces and duchies all at once.   Unfortuantely, the uber turn doesn't happen on the same turn every game.   Consider 2 games, one where the uber turn happens on turn 15, and one where the uber turn happens on turn 16.  Average just those 2 games, you will find that the average score for just half of what the actual score is for those turns.   As a result of this effect, it never looks like the engine is ahead, even though it usually wins.

When averaging turns measured from the beginning of the game the end game data does not lined up.   It becomes difficult to see how a strategy is adjusting to finish the game.   Instead of counting turn 1, turn 2, etc, lets line the data up differently!   We will play though the entire game, find out where the game ends, call that turn 0, and count backwards to label the other turns.  The second last turn is -1, third last turn -2 etc etc.   Now the end game strategy becomes apparent.   


You can easily see that the engine shoots for an uber turn and usually wins on average 35 points to 25 ...   

Now while this approach is good for analyzing the end game, it would be unsuitable for analyzing opennings.  (using similar logic).    There are two approaches to counting turns - counting turns from beginning and counting from the end of the game.  Both are needed to comprehensively understand a strategy's performance

Full comparison report attached.

[florrat]

Nice report! Yeah... When thinking about it, it is not strange that for megaturn strategies the average number of VP in your deck during the game is very low. Nice how graphs can look as if they give wrong information about the game.

Could you make a graph of #vp gained that turn? I think that would describe the "truth" pretty accurately, while not requiring to count from the back of the game (not that there's much wrong with that, except that it may be a little harder to read). It will probably be very similar to the very bottom graph of the html-file.

About analyzing the opening: I think that no graph showing victory points accurately describes how well an opening goes. You want something like either average money per turn or average #cards gained per turn, probably both. (fun fact: during your simulations, turn 12 is the only turn where BM bought a card in every game)

[Schneau]

This is great stuff. Another problem you might be experiencing is that the game ends when the Combo Engine hits. Because of this, the points bought that turn only count toward one turn during the game, where BM's average victory points per turn is slowly added to throughout the game -- thus, a Province bought on turn 7 is going to factor into turns 7, 8, 9, etc. I wonder what would happen if you instead included the final score in the averages of future turns even if the game had already ended -- so, for turn 20, include all scores at turn 20, even for games that ended before turn 20. This method might see the Combo Engine pull ahead of BM at some point.

[Sparafucile]

About analyzing the opening: I think that no graph showing victory points accurately describes how well an opening goes. You want something like either average money per turn or average #cards gained per turn, probably both. (fun fact: during your simulations, turn 12 is the only turn where BM bought a card in every game)

   I agree.  Please see the attached report of the original post.  It has a much more complete set of graphs depicting what was going on at the beginning of the game.  There are too many graphs to include pictures of them all in my writeup