Both phrases have the same meaning. I would tell a friend of mine to "come over" or to "come by" if I want them to come to see me. The only difference is you could use "come over" if you want someone to spend the night at your place while "come by" would never mean that. "Over" would thus mean you are wanting that person to come overnight. But in reality, they just mean the same thing. Another phrase you may hear that is similar is "come on by" or "come on over". Some people in the southern United States, including myself, add the word "on" to the phrase. It doesn't change the meaning of it, just adds to the flow of words. Hope this helps! Keep up the good work.

Another colloquial phrase could be, "drop by." Example: "I'll drop by your house later."