We tend to think of "handsome" on one side of a line, and "ugly" very far away on the opposite side. But sometimes things return to the opposite side of the spectrum. For instance, "very late at night" is the same thing as "very early in the morning." Here, the person is SO handsome that it makes him look strange and inhuman, and it is almost the same as being ugly.

Not enough context to get certain meaning from it. At a guess, it means he is so good looking that is actually a negative. ie. His attractiveness actually discourages or puts off people for some reason.