I’ve always found it odd when people repeat the object: “You too.” The reason is that the object isn’t in question; it’s the subject that’s in question.
Look at it this way: What do you say when someone says “I love pizza”? The technically correct response is “I do too”. So you’ll notice that the thing in question is the subject: Do I love pizza as well or not? Saying “Pizza too” is absolutely meaningless, because what's in question is not what one loves, but rather who loves pizza. You can say, “I love burgers. Oh, and pizza too!” Here, the object of your love is in question: What is it that you love?
Likewise, when talking about people, “You too” seems like a continuation of me talking about the people I love. “I love him, and you too!” In other words, “I love him, and I love you as well.”
In addition, when someone says “I love pizza”, it’s perfectly natural to say “Me too”. This might be technically “incorrect” from a prescriptive grammar standpoint, but English (especially spoken English) does not follow strict prescriptive grammar rules. “Me too” is just less formal than “I do too”, but it still refers to the same thing: the person speaking.