It's tough, and I don't think there is a consistent way of saying what is or isn't a number across all of mathematics. You could say something like "any structure that allows arithmetic in a way such that it contains the natural numbers", and that would cover a lot, but it would also make you call matrix rings or rational functions numbers, and arguably even weirder things like isomorphism classes of surfaces.

I think the process of calling something a number is more inductive - like, you start with the natural numbers, wich are the "original" numbers. And whenever you encounter a problem that you cannot solve using your existing classes of numbers (what is 2-5? What is 2/9? What is sqrt(-1)? How many elements does the set of natural numbers contain?), mathematicians are prone to call the structure they come up with to solve that issue a class of "numbers" again.

Which would make a number "any element of a structure that solves a general problem for which the natural numbers provide solutions in special circumstances".