I think your intuition is (probably) pointing to something real! Prediction isn't just about calibration, there are two quantities: calibration (which is that your x% predictions come true x% of the time) and boldness (which is how far your predictions are away from the baseline). People don't differentiate the two, in fact there's no established name for the latter; boldness is just something I've made up. An 80% prediction is often (though not necessarily) bolder than a 50% prediction, so probably this is what your intuition is based on.
E.g., say two people take the same sheet of paper that has 100 statements written on it. One person flips a coin for each; if it's heads, she bets that the prediction comes true; if it's tails, that it comes false. This person is necessarily perfectly calibrated, but the boldness is 0 (the procedure literally yields perfect calibration for an arbitrary set of statements). The other person sorts them into two buckets and bets 80% TRUE on each in bucket 1 and 80% FALSE on each in bucket 2, and then is "right" every time. Now, the second person isn't perfectly calibrated, sheshould have assigned more than 80%, but her predictions have much more boldness, so overall they may be more impressive.
I think the reason why people struggle with this concept is that baseline isn't an accessible standard. But it's clearly a real thing. Like, I can't tell you what exactly the baseline is for "at the end of 2022, ETH is worth exactly 1502$ and 35cents", but we all understand that it's very low, definitely <1%. And we intuitively understand that successfully deviating from the baseline is impressive.