Why is it correct? Because 'joining' is a gerund. A gerund is a verb form which functions as a noun, so it needs a possessive adjective ( not an object pronoun) before it.

If it helps--I'm a lawyer in the American South and speak professionally all day long. Most people would find "my joining you" to be grammatically correct but they may think you're kind of posh or snotty. It wouldn't sound natural. This is such a fine grammar line that I don't even know why "my joining you" is grammatically correct other than the fact that it sounds better. Just go with your gut and relax.

Both are fine! I would naturally say "me." "my" strikes me as more formal, as an American, and but I remember reading that it's widely attested in UK speech. To add a "non-answer" on top of my answer, I would say that I don't often use that particular construction, and I think it would come across as awkwardly formal if I did. To communicate the same thing, I usually say, "Would you feel comfortable if I joined you?" or simply "would you mind if I joined you" (because asking if someone feels comfortable usually implies that they might not feel comfortable, and maybe you don't want to imply that).

I'm unsure about the official grammar but in the US you would typically say/hear "Are you comfortable with the idea of me joining you". Know that this sounds a bit timid or shy. More likely you would say something like "Would it be alright if I joined you" or even simply "May I join you"

Are you comfortable with the idea of my "brother" joining you? Are you comfortable with the idea of me (myself) joining you?