Your assumption is basically correct. "He is from Japan" specifically is saying where is from. "He is Japanese" can refer to race/ethnicity or nationality depending on context. Nationality and ethnic background are two different things obviously. I know many people in the US who could be described as both "Japanese" and "American," for example, but you couldn't say they are from Japan.