Hermit is a card that will be much more powerful in the context of Dark Ages than in every-card Dominion. The reason being, targeted trashing. Trashing without decreasing your hand size is cool, and will be a decent use of the card outside of Dark Ages (I've decided "low-impact trashing" is a thing). But, when you're trying to hunt down and kill that one On-Trash card in your deck, Hermit can see over half your deck on average. This gives much better odds of being able to hit your target than, say, Remodel which can only target one of the other four cards in your hand.
In short, Hermit is not a actually a trasher. It is a guided missile for On-Trash abilities. This and Madman should be considered its primary purpose in life.
(This seems kinda obvious in retrospect, but I didn't notice anyone else explicitly mention it. Apologies if you're invisible.)