You should consider Island. Any decrease in your deck size is very good here. I think I would open Island/Silver, Island/Watchtower, or Baker/Watchtower. The Wandering Minstrel looks like a weak opener.
Turn 4 should be Wandering Minstrel over Fortress, until the Wandering Minstrels run out.
I would take Goons over Baker on Turn 5.
Buy some Advisors! By turn 13 your deck has enough money. You should be buying Advisors/Cartographers instead of Bakers because your only goal is to play as many Goons as possible, the coin tokens are not so useful.
Your turn 17 Duchy is way too soon. Bakers and Goons still have 3 cards left, and they are expensive so the game won't end right away.
Remember, Goons games are going to last waaaay longer than other games, and Goons/Watchtower means you have an endless supply of VP to catch up with, as long you keep your deck under control. It would have taken a long time and some luck, but even when you resigned you were not out of the game yet. Though your opponent could probably 3 pile before you caught up.
