I don't think it's that backward to be honest, if I am a wee personified time before 3, looking towards it, the minutes following 3 are at the back of it (depending which way 3 is facing I suppose, but let's say the front and back are relative to me) so 3.10-3.15 (what most people seem to mean by the phrase) is just behind 3, or "at the back of".