The first is wrong. The expression is ‘to spell something out’ which means to explain it thoroughly in simple terms. The options would be I don’t think it was spelled out properly. I don’t think it’s been spelled properly. These have similar meanings with latter stating more explicitly than the former that there is a current effect of the past action.

It doesn't sound natural to me. To what does "it" refer? If "it" refers to the word that has been misspelled, then the sentence does not make sense because words cannot spell. Only people can spell. If you say "spelled" then you need to use passive voice: "I don't think it is spelled out properly".

Do you mean SPELLED literally, or do you mean not explained. I don't think it's been spelled out properly/correctly. = It has not been explained well. I don't think it's spelled correctly. = (a word is misspelled e.g. THENK instead of THINK)

The first one is far more natural for an American English speaker, the second one may be more common for a British English speaker.