You could say the same about Forager -- trashing a Gold also helps your opponent, but it may help you more depending on how many Foragers you have.
Well, you hardly ever trash Golds for Forager at all, and when you do, it's for tactical reasons. Either you just need the extra coins or the buy now at more or less any cost, or you temporarily have more Foragers than your opponent. In the latter scenario, your opponent can easily just buy more Foragers if the advantage is big enough to make a relevant difference, but usually he won't, because it's not that huge. Or you just want to get rid of the Gold for whatever reason.
Alt VP cards are also bought for tactical reasons (buying a 2-point Gardens in the late game because you didn't get $5 and you need any points you can get), but that's different from the Forager scenario because nobody has to do anything special for that: you just buy the card, get whatever points you get from it, and that's it. It hardly matters what the criteria is for getting the points, as long as it's somewhat balanced. But when you're actually trashing cards from your hand to increase the VP value of Silk Rat, you do that because you're making a long-term commitment to a Silk Rat strategy. Meanwhile, your opponent can do whatever he wants. If it was a Gardens instead, your opponent then has the choice of buying some Gardens just for the purpose of denying them from you, or focusing on the Provinces or something and letting you have all the Gardens. But with Silk Rat, now that you've done all of the work for increasing the VP value already, it's rather trivial for your opponent to just get a few Silk Rats himself, and then whatever work you put into making your Silk Rats worth a ton of points for you also helps your opponent while he can just focus on the Provinces without helping you at all.