'Japanese teacher' sounds more normal. We usually make a compound noun with the word 'teacher' and the subject that they teach: a history teacher, a geography teacher, and so on.
We'd only normally use the more unusual 'teacher of ..' structure in two cases:
- if the subject is long (more than two or three words)
- to avoid ambiguity
In the case of Japanese, you might want to say 'a teacher of Japanese' for the second reason, to avoid confusion about whether you're talking about the person's nationality. In spoken language, that shouldn't be necessary, though. To a native speaker, the difference should be clear: 'a Japanese teacher' with the stress on 'teacher' is a teacher of Japanese nationality, while 'a Japanese teacher' with the stress on 'Japanese' would generally be understood as meaning a teacher who teaches the Japanese language.