The reason is to slow down its travel. In e.g. a 4-player game, if you play a Wanderer, it's going to be at least three turns before you see it again. If the player to your left gains it, it's you might have it on your next turn. I guess in a 2-player game you might see it on each turn anyway.
Wanderer: Action, $3
+4 Cards. The player to your right gains this card.
Normal gain, as in goes to the discard pile of the player to your right? Unless you're all cycling through your decks pretty quickly, the receiving player won't play it again right away and it won't move super-fast, so I think player to the left might be okay. Nothing wrong with the right though.
Being terminal, there's the chance it could get "stuck" in the deck of a player who maybe already has a bunch of other terminals they'd rather play? Maybe make it a super-Lab (+3 cards, +1 action) instead of a super-Smithy, so that it fits into almost any deck?
It could also spur politics of the form, "I'm not playing it because the player to my right is winning and I don't want to give it to him." I guess you could force the movement with something like:
Action - Duration
+4 Cards
At the start of the player to your right's turn, that player gains and plays this card.
But that version definitely needs an on-buy bonus because the buyer is the only one who ever has to play it terminally or have it in their deck.