My husband had tough time like you in Japan, so he joined some igo clubs, gokaisho, as he really liked playing it.
He also went to an English-speaking club, Eigo-cafe, which was a kind of meet-up held at a cafe. Most members were Japanese people who wanted to have conversations in English. Actually, you may sometimes need to speak in Japanese depending on their English skills. I am pretty sure that you can be highly welcomed by them because of your English skills. You can make new Japanese friends there.
Nick wants to meet people and have an occasion to make friends. You didn’t write your opinion about that. Is it possible for a non Japanese to make friends among natives? I heard that it is almost impossible.