Actually, assuming it doesn't hit two estates (much less likely than hitting a potion or a silver), it will reduce the value of turns 4 and 5. The average value of 7 coppers , 1 silver, 1 potion, and 3 estates is $.75 per card. The average value of 6 coppers, 1 silver, 1 potion, and 3 estates is .75, while the value of .727.
The average value of 7 coppers, one potion, and 3 estates is .636.
When getting that first familiar, every copper counts.
Also, if you go ironworks in response, that makes things worse on your end, since it slows down your initial ability to buy familiars as opposed to going straight for a mining village, or silver.
The smart response would be to go horse traders. Since a horse trades + potion hand guarantees you a familiar, and its reaction would give you a good advantage over the pirate ship player. (Luckily, most people who go for familiar spams don't seem to realize the value of horse traders).