Hi 辉
Your sentence is half right, its a mix of two ways you can say it:
'what does a man wearing one have to do with it'
'what has a man wearing one got to do with it'
it means - what relationship is there between the fact that the man is wearing something (like an item of clothing, for instance) and the subject about which we are speaking.
So, if you were talking about motorcycling, you might ask that question in response to the statement 'I saw a man wearing a scarf on a big Harley the other day'.