People may usually say "I'm talking to you" when they mean to say "I'm talking with you", and thus both are accepted colloquially.
However, there is a difference, and it's relatively obvious if you know the common definitions of those prepositions.
"Talking to you" means that I am the one talking, and you're the one listening. It's almost identical in meaning to "talking at you", except that "talking to you" has the implication that you could talk back if you wish (and is thus a bit less rude).
"Talking with you" means that we're both talking to each other. That is, we're having a conversation. You can't talk with someone if they sit there without saying a word.
It's easiest to conceptually differentiate these by applying them to non-human objects. Let's say you have a cat. You can talk to your cat. You can talk at your cat. But you cannot talk WITH your cat, because cats obviously don't speak human languages.