It's not enough to just play Cave every turn, you need to just-as-reliably buy expensive cards to set aside. That said, a Golden deck (Gold, Gold, Silver, Cave, Cave/Province) does produce the quadratic payoff - setting aside a Cave (later, Province) every turn.
This makes me think that the card must care about uniques. If each Province on the mat makes Cave worth another ~1VP then the Golden deck (with 3 more Caves on the mat) produces ~10VP/turn. That's too much, it would almost always be the optimal strategy. (If a Province on the mat isn't worth even 1VP, then Cave just looks pointless and weak.) i.e. the first Province on the mat has to be worth it, but the duplicates can't count.