It's actually intended that way, I like the idea of a cheap card that has this downside, but can be comboed easily with highway, bridge, peddler, quarry, watchtower etc to remove it. The card isn't good enough to break the game otherwise (it's cheap woodcutter plus) on purpose.
Maybe this card isn't the best one to chain this downside to. Something like +1 action +$2, gain two cards costing less than this when you gain it for $2 could work, and just price this at $4. But then it's a more expensive woodcutter if there are no potion cards, it becomes impossible to price.
Also, with the phrasing the way it is, does it allow you to gain curses if the curse pile runs out (similar to torturer)? Because that was the intent.