Yeah, I've come around to thinking this recently. I do think harvest is the worst $5 card, which is really weird, because it isn't really bad. In fact, it's usually not even the card you'd swindle a $5 into most in most cases - even if we wouldn't allow duchy.
But you know, I'm pretty sure I'd put royal seal very close after it, because it's also just not all that useful. I mean, it's always fine, virtually never worse than silver, but... I mean, you can look at stash similarly to it, and which one is better? Usually RS is a tiny tiny bit better, but the problem is that in those few cases where stash is better, it's incredibly powerful.
So if I made a list today, I think it would probably be Harvest, Rogue, Royal Seal, Graverobber, Counting House, Stash, Pillage, Mine, Saboteur, Tribute, Cache, Outpost, Explorer, etc. or something thereabouts. I really don't have a good feel for Dark Ages though.
Tribute, the problem is, well, where is it good? I mean, it's really good if they are swimming in Harems, but how often is that? And so, ok, 4 cards is magnificent for $5, but it won't do that often (and it comes with the drawback of skipping two of their dead cards - already I am not sure that this is better than Council Room); 2 cards and $2 is really good for money, but you usually have a better payload in anything resembling an engine. 2 cards and 2 actions is obviously really really good, but again, they don't often have so much green. 2 Actions and $2 is worse than festival by a buy, and that's okay, but nothing super special. 4 actions is great if you have a draw-to-X card. But I mean, it is village village necropolis, or NVx3, picking up the third and starting with an empty mat. And well, if you have lots of terminals this is good, but the terminal density you'd need for this, you are going to have big chances at very dead hands. And the problem with counting on it for actions isn't so much that it's unreliable (though that can indeed be vexing) as it is that, well, in order for it to be reliable, they have a big action density. And if they have big action density, they probably have a different village to support, and getting that and something else on 5 is almost always going to be better than this. Add in that sometimes it's going to be terminal silver, occasionally moat-without-reaction, and (albeit rarely) necropolis, and that it does nothing for you at all on curses (and hovel), and it's pretty bad, even while very often it gives you a benefit that would be too strong to put on a $5.
Harvest... terminal gold-ish for 5 is actually not that good. And usually when it is, you are building a draw-your-deck engine, and this has the unfortunate property of not producing any money when your whole deck is drawn.
Counting House has so many nice things going for it, almost all of which are about 20% too complicated to actually work and be worth it. But it's an enormous counter to mountebank, and it has some nice marginal uses, particularly with high-sifting engines. Also not terrible in grind-to-a-halt kinds of games - goons and inn come to mind. But with some sifting, buy, village, can often be worth the cost. But very rarely by very much, and there's usually something a bit better.
Contraband can actually be excellent when you need +buy, and really almost anytime that there are lots of options you wouldn't mind. Especially if you can T4B it later. Okay, it's not too often, but it can really shine, oh maybe 15-20% of the time?
Cache is excellent for slogs. Really really great - except after long, you are buying green instead. And it's okay for big money. Just usually, you wish there was something better to do for $5. Of course, it also combos really nicely with trader and has a mild but in-a-pinch useful watchtower ticket.