Seems to be a definite difference. Drawing from the top of your deck is so commonplace that "+1 card" makes sense. In fact, you do it five times every turn. (You do not "draw your hand", you take +1 cards five times.) The other ways to put cards in your hand (Counting House, Mine, Beggar, IGG, etc.) are infrequent and specialized. Discarding and getting things out of your hand, on the other hand (heh), is extremely varied. You can discard from your hand, you can play from your hand, you can place cards from your hand on top of your deck, other players can discard a card for your, other players can make you discard cards of your choice, you can discard cards from play, you can discard cards from the top of your deck, you can place cards from your deck into the discard pile (not discarding), etc. You can also trash cards. (Why not make "-1 card" mean trash?) Many cards respond to discarding with special rules (e.g., Tunnel). I feel like "-1 card" just isn't precise enough.
It also breaks some symmetry. +1 Card is always a face-down card from the top of your deck. You don't get to choose it (though you could have set it up to be a particular card). If -1 card meant discarding from hand, then you have to make a choice about what card. And you discard to the discard pile, not to the top of your deck. The actions are actually different, not simply the reverse of one another. And as I believe someone else mentioned: for any way you define "-1 card", it does not hold that +1 card -1 card = -1 card +1 card, so commutivity is broken as well.
Edit: Oh, also, you can discard your hand, but you can't draw your hand. Or at least they're not the same. Discarding your hand during cleanup allows you to discard in any order you want. (Or do you discard simultaneously with any rearrangement you want?) During your cleanup phase, you draw five new cards, one at a time.