I can understand your feeling quite well.

Whenever I speak with Chinese friends, and my Chinese seems not to be understandable for him/her, I always find myself searching Korean words which I learned in advance unconsciously. Although he/she doesn't know Japanese, At least they are interested in Japan (so that they speak with me). But Korean is utterly out of their field. So the conversation gets more and more confused. And when I spoke with a Vietnamese friend for the first time, then, I found myself searching Chinese words!

Curiously, English doesn't appear in such cases. Probably my subconscious is noticing that the person I'm speaking to is an East Asian people, and cannot understand English just like us ourselves. But they cannot understand Japanese at the same time. So I don't have to use Japanese, cannot use English...Maybe this is the limit of my subconscious. And it starts searching words "not Japanese and not English" with full speed. I also cannot control this for now. I think the only solution is to stop "love affair" and concentrate to one language. But it seems very difficult for me. Because the world is full of wonder.

There's another curious phenomenon. After speaking with foreign people with other languages, I always find myself forgetting how to speak standard Japanese (Tokyo dialect), and speaking Kansai dialect (which is my mother tangue) in the middle of other district where no one speak Kansai dialect. Is it universal, too?

Thank you for sharing. It's good to see that it's not my brain malfunctioning and that it's a common phenomenon. 

I think you're right, concerning the 'affair'. I was hoping I could keep learning both at the same time.