The problem with Followers is that, though a strong card in its own right (it would surely cost $7+ in the supply), it has a huge antisynergy with Tournament.
But it forces that antisynergy onto the opponent as well with its junking and handsize attack. I do find myself in situations where I really don't want it sometimes, but you can't be too hipster and always insist on skipping Followers.
Handsize attacks don't make it harder to fire Tournament though. Junking does, but you junk yourself too. The VP swing isn't too helpful unless you're on a very sloggy board where tournament isn't as powerful and terminal draw sucks.
It can if the person being attacked has either Province of Tournament in hand but not the other. Keeping Province increases the risk of stall, and keeping Tournament risks having them get blocked and having a completely dead turn.
In the presence of trashing, the player with Followers has a slightly easier time trashing the Followers junk thanks to having the +2 cards off Followers and not starting with a 3-card hand.
The VP swing might not always matter, but sometime you might end up slacking on proper engine building to actually get the prize you want first. Getting a bunch of Tournaments is not equivalent to having an engine, especially if the Tournaments start getting blocked. At that point, you have something of a slog.
If you have T but not P in hand when the attack hits, then in most cases you are going to keep T, and whether or not your draw kicks off will be the main determiner of whether you get the P, which would still be the case with 5 cards. If you have P but not T, then you may well discard the P for kickoff and then draw a T, or keep the P but then not kick off and not draw the T, so it is a bit more of a problem there, but I do not think it is that much of a hindrance anymore than discard attacks normally are.
Do not get me wrong here: I am not arguing that Followers is not a great card and is not a good prize to go for in a lot of situations, I just think that the fact that it is a prize weakens it more than the same fact weakens any of the other prizes except for BoG, and I think that means it is not the best prize to aim for first in as many scenarios as is often argued.
At the point you get the first prize, you are either already in a super-trim state and lined up your T and P because of that, in which case you probably want Princess, or you lined up by volume of Ts in what will then be a slightly messy cantrippy deck that you haven't transitioned into an engine yet, in which case terminal draw is nails for you, the self-junking kills your transition to engine, and you really would prefer Trusty Steed to help you transition to engine. The latter situation is probably better for Followers, as the long, junky game you will then get makes the VP advantage it gives more potent, but your opponent will then probably get Trusty Steed and may well be able to cycle faster and deal with his curses better than you can deal with your estates. That is the other big drawback of Followers compared with Princess. When Princess is good, the consolation prize your opponent gets is a Trusty Steed they don't really need or a Followers you can deal with. When Followers is good, the consolation prize is a Trusty Steed that will be nearly as good as Followers.