I've been playing a decent amount of Base-only games, and I'm surprised at how often people are messing this up.
It's okay to buy Gardens over Duchy when your deck has fewer than 30 cards. As long as your deck is 30 cards [by the end of the game, you'll break even. And on the off-chance the game turns into an endgame slog, your Gardens could tick up to 4 VP. If you know you'll hit 30 cards, buy the Gardens instead.
You judge cards not by how good they are right now, but by how much value they'll give you over the course of the game. It's easiest to get an appreciation for this by opening Monument. The 1st Monument play only gives 1 VP, which is okay but not great. By the end of the game, that Monument is usually worth 4-5 VP in money-based games and potentially more in engines.
Figuring out when to buy one card over the other is hard, but for Duchy vs Gardens it shouldn't be hard to make the right choice.
(In similar veins: it's okay to buy Silk Road before you have 12 Victory cards, and it's okay to buy Fairgrounds when they're worth 2 VP, as long as you have a plan to meet the requirements in time.)