I don't think this is a rule. The first two examples you give definitely feel wrong to change, I am not sure that I would think a thing of the third one, but more than that there are a bunch of examples where it doesn't really matter:
My old dirty hairy brush
My old hairy dirty brush
My dirty hairy old brush
My hairy dirty old brush
My hairy old dirty brush
My dirty hairy old brush
Now, which one of these is right? A couple of them sound a little better to me, but not anything really clear.
Now, numbers pretty much always seem to come first - but I think that's because it's principally a noun. The class of word that French, Italian, etc. belong to (the name for which escapes my mind at the moment) *typically* feel like they should go last, but I don't think this is a hard-and-fast thing: given a lineup of 7 old men, saying "The French old man is the one I'm looking for." seems ok, though admittedly this has more to do with collapsing of "old man" into a single concept than anything else, and "Old French man" definitely seems fine and perhaps more normal.
I think for the most part, it has to do with how much differentiation each adjective gives you/how normal the combination is, and I don't think it's a hard-and-fast rule. Probably a style-guide kind of thing.