I think this question is deceptively simple. You will always want the SMALLEST deck you can possibly have that will allow you to win. Yup, simple as that.
The catch is, with each table you play, your answer will change due to what kingdom cards are available to you. I will also say that the only cards you can never have too many of are +Card, +Action cards as they will never bog down your deck*. Also, there is always "the worst card" in your deck. You could have a deck of Gold, Gold, Gold, Gold, Gold, Plat., and the worst card would be Gold. So, never think there isn't room to cut something out.
Even with Gardens you should only go above 50 cards if you NEED them to be worth 5 VP to win. If you can win with them at 4 VP, then do so (and you'll probably do so more quickly).
*It may be true that they don't bog down your deck, but before buying one, you should ask yourself if a Silver (or Gold) would be a better purchase.