If someone asked the initiating question, they probably (or should have) said: "Would you like to ACCOMPANY us?" The word "accompany" means "to go with".
And really, you could get away with simply asking "Where?"
This whole exchange seems very formal. In the United States, it would be more like this:
A: "Do you want to go with us?"
B: "Where?"
-or-
A: "Do you want to hang out with us?"
B: "Depends ... where are you going?"
All of these are very colloquial, everyday speech. Don't try to make them grammatically correct. They aren't. But they are the way this exchange usually happens.