Saboteur is quite possibly the most annoying attack card in the game. Not only does it target only the cards you actually want in your deck, it keeps looking until it finds one, and destroys it. Most aggravating if that was a card you needed, even if you could turn it into a cheaper one that was at least somewhat useful.
Enter sab's kryptonite: The fortress. If you have fortresses in your deck, you WANT your opponent's sabs to hit one of them, because this is what will happen:
1. The attack is negated, without the need for a moat or lighthouse.
2. You have the option of gaining a $2 card (because you technically DID trash the fortress).
3. You add a card to your hand during another player's turn.
4. Which just so happens to be a village.
(Note, BTW, that countering a knight with a fortress--that just sounds so cool, BTW--gives the attacked person all of these benefits except for #2.)
If fortress did #1 alone--blocking the attack--it would be worth the investment. But by giving you all these "little" benefits that can really add up, fortress turns saboteur into an outright liability. Sabs are rather weak as-is, but with fortresses on the board, they can become as bad as scouts. Maybe worse. In short, if both cards are in the supply, don't even think about picking up a saboteur unless there is a very specific reason for doing so.
