Because humans inherently hate things that are different to them and skin pigmentation is probably the most striking difference between people of different races.
We then became civilised and got educated to a level where we know it's absolutely thick to hate people because of their skin pigmentation. Unfortunately, some people are still too thick to get this.