It simply means that you have been saying something - usually something bad - about a person without his or her knowledge.
Example:
You are sitting in a café with your friend Fred and you have the following conversation:
"You know that new girl in Marketing?"
"Who - Alison?"
"Aye; that's the one ..."
"Well, what about her?"
"I don't think she'll last long in that job: she's lazy!"
In that example, you have been talking about Alison behind her back, which is very bad form.