'Interlocutor' is an extremely rarely used word in English. To the point where a lot of native speakers will have never heard it before, let alone know what it means. It's a term which is primarily used in linguistics. So no, that part is not natural. 

'Don't take it personally' is perfect but I'm not sure exactly what sentiment you're trying to convey in the second part of the sentence. You weren't talking about the person? Or you weren't talking to the person? I have a feeling it's the former but please feel free to correct me if I am wrong. In that case I'd say 'don't take it personally, I wasn't talking about you.'

(I'm not a native speaker)

I think it could be: "I wasn't referring to you", "I wasn't referring to you personally/specifically". Or, depending on what you meant, "I wasn't talking to you"

"Don't take it personally" is correct. I think you can also say "don't take it to heart" or "don't take it wrong".