Often used when talking of love, You belong to me is a possessive statement, for example: Your my wife, you belong to me, your mine.(negative) You belong with me - example: You belong with me, I belong with you, we belong with each other - I want us to be together for ever. You belong with me is often used in song lyrics when expressing love to another person.(positive)

In the context both means the same thing. But I think "you belong with me" is not a right sentence. "You belong to me" means that, that thing literally belongs to that person. But if we see at "you belong with me" it mean something like, that thing is that persons and looks better him/her and should be with him/her. But native speaker can explain better.