Other shadows are 10-11 o clock..
In other news, English idioms don't translate well to other languages.
"Five o'clock shadow" refers to beard stubble. The idea being that on a thickly bearded man, he'll have stubble by the end of the workday after shaving in the morning