I agree. I feel like often there's a Duke on the board, which I have no care to pursue. But then my opponent buys a Duchy... and as soon as he does, I feel like I have no choice but to start buying Duchies, because if I don't, my opponent will end up more points from Duke/Duchy than I can get from Province.
1. So, the question to ask yourself here is: why do you have no care to pursue the Duke? If it's a board where Dukes are going to beat provinces, then this was a mistake to begin with - it doesn't become a mistake when your opponent buys the first duchy. If it's not such a board, then you should relax! Your opponent has chosen the non-dominant strategy, and you'll probably win.
The following is a simple heuristic, so forgive me if you've already thought of it! I find it very helpful, you might think it is a tautology:
A good question to ask yourself in these circumstances is: "why did I decide that provinces are better than Dukes in this game?"
The answer is usually of the form "I worried that if I went for dukes, he could do x."
Now, when you see your opponent buying an early duchy, do x.
2. On similar grounds: try playing some rush strategies, especially when you're not sure whether they are dominant. Even if you think, OK, IGG + duchies is boring, and I would prefer never to go for the rush when possible, you still need to learn how to defend against the rush. The best way to learn defense against a strategy is to execute the strategy yourself and pay attention to what your opponent does and, more importantly, to what you're worried about your opponent doing. For instance, try going for workshop/gardens on a colony board or for IGG with watchtower on the board - corner cases like this.
3. A general Dominion principle (which doesn't always hold) is: go down with your ship. Maybe it's a board where the opponent's vault + duke strategy has a 60% chance of winning over your, I dunno, fishing village festival watchtower deck which aims for provinces, and you don't realize what's gone wrong till he's bought the first duchy. OK, whatever, sucks that you made the wrong choice, but a 40% chance of winning isn't so bad. It's probably better than a non-strategy which buys some watchtowers and not enough festivals and then freaks out and has to repeatedly decide between buying a duchy with $12 or buying a province and losing. [this principle is not meant to imply that you should not buy duchies - you have to prevent his dukes from being worth 6+, but only that you should not jettison your original strategy in the course of doing so]
4. I agree with everyone who said the different rush strategies are fundamentally more different than they are similar. Still, I think my bullet point #2 above is the best advice for defending against all of them (and against every other type of strategy).