Yes, in fact, it is the ONLY normal way to ask this question in everyday English. The supposedly 'correct' forms 'Whom were you with?' or 'With whom were you?' are never used by native speakers in modern English.

It is fine in everyday speech. The grammatically correct version is "Whom were you with?".

Absolutely, feel sorry for the one who needs to reply

I think that anyone you talked with would understand you.

Yes, it's correct. :)