I think I like this card... but I suspect it would be too powerful at $4 because it's non-terminal and has such powerfulcumulative effects -- in many games, this would be more powerful than Bridge.
The reason being that since the point of the game is Victory cards, this allows all sorts of tricks like the one you mentioned. With two Chamberlains in play, you'd suddenly be able to use Rogue and Knights to trash your opponents' Provinces or Swindler to turn Provinces into Golds. And Rogue or Graverobber to pull those Provinces back out of the trash and into your deck. Yes, you can pull some of those tricks with Bridge, but it's much harder to do so since it's non-terminal, and the Swindler trick wouldn't work at all, since Bridge reduces the cost of ALL cards.
Same with all the TFB cards... a couple of Chamberlains, and all of a sudden Forge, Remodel, Expand, etc... have a much easier time churning out Provinces...
Still, I like the card -- I like those kinds of tricks -- but it would need playtesting to determine cost to see at what cost it is balanced.