Express it in a positive, not a negative way - unless you REALLY don't like this person. "Oh, sorry! I was talking to him/her." Smile when you say it. That makes the person feel at ease, as it it was just a common mistake that anybody would make.

Hmm, that is already polite. Maybe preface it with an apology: "I'm sorry, but.I was actually addressing someone else". Not much better, maybe. If you had needed ruder ways to say it I could have been much more helpful :P

Ooops, these days my eyes don't work good, they turn in wrong directions, i guess. that is why you thought i am with you,sorry for my faulty eyes,i will go to doctor office a.s.a.p! :p

Sorry, I meant that for......... Thanks for your answer but I was actually speaking to.....sorry! (Taking the blame can help too) "sorry that's my fault, I was asking....... Thanks anyway!" Something like that. Use manners, they go a long way in English :) (My daddy had to learn all of this the hard way, in the past he'd just say, "I wasn't talking to you" and think there wasn't anything wrong with it. He doesn't like to take the long way around, when he can draw a straight line, but "people skills" help a lot in life. His claws can still come out though when he dislikes someone :) That's my daddy!! Miow miow!!!

I would say, "Don't you agree that it was a such lovely chat with XXX?"