You know, every time I've tried to pull off transmute/duke, I've lost. I keep thinking it's a great combo, but it just doesn't work. Here are the problems:
1. What do you buy with 5? In an ordinary duke game, I think the answer is unambiguously that you should buy duchy until the duchy pile is empty, then duke (unless the game is about to end and you have at least 5 more duchies than you have dukes).
However, in a transmute game, once the duchy pile is empty, your transmutes are worthless and it's quite hard to buy a duke. So maybe you should buy them earlier? But then you've got all these dukes clogging up your deck, which brings us to problem number 2:
2. It's not like your opponent can't get duchies the old-fashioned way, i.e., by buying them. And even an exceptionally dim-witted opponent will notice very early that you are going for the dukes.