Best Answer - Chosen by Voting
People have always been judged by their appearance. It takes a lot longer to know a person and objectively judge them on their internal merits than it is to take a quick gaze and apply some useful stereotypes. There's just too much effort unless they're family, close friends, or you want to sleep with them.
It's not fair, per se, but it's definitely how humans work. I think some cultures have made strides to overcome this bias, but to say that it's not happening at all (that is, judging people on their physical appearance) would be absurd.