Poacher is often just as good as Peddler, but sometimes it is worse. Only in extremely rare edge cases is it better. That makes it worse overall.
"Extremly rare edge" is an exaggeration. Library and Tunnel directly gain from it significantly whereas if you already green or have junk in your deck discarding a card is neither beneficial nor advantageous.
So while I agree qualitatively, i.e. that Poacher is on average worse than a hypothetical Peddler at 4, I consider the strength difference to be very small.
The main reason is that once a pile is empty you usually either start greening or still build up a strong engine. In the former case you can discard Victory cards whereas in the latter case it is fairly insignificant whether you have a Peddler or an Oasis in your deck as you strive for more powerful stuff in your deck..