I think a lot of times players on both side of the table don't really want a certain card to be in the game. It'd be nice if there was a feature that let both players play the game without a card being in the game if it was unanimous. Agreeing not to dogma the card if it comes up doesn't work that well for a few reasons.
You could make the card be the achievement, perhaps, or at least that works for Clothing, Industrialization, and Gunpowder (or X, not here to debate what the most unfun cards really are, just suggest a feature for when it's -unanimous-)
If the UI would be cluttered, you could just throw up a bunch of popular cantidates for removal.