Somewhat the same. "What's the matter with you?" and "What's wrong with you?" can both be perceived as rude and would be more a problem about the person rather than an issue they're having. "What's the matter?" or "What's wrong?" are more polite. "What's up?" can mean the same thing but can also be used as a greeting. "What's the trouble with you?" or "What's your trouble?" are grammatically correct, but awkward and not used. Closest would be "What troubles you?" but even that is uncommon.

Lots of difference. what's the matter with you? Use as in "you are crazy or weird" what's the trouble with you? Use as in "What is wrong? Can I help you at all?" what's your trouble? Never heard this one. what's up? How do you do? What's new? (But casual) what's wrong with you? Can I help you? or "You are crazy!"