Well... it's a bit odd, because I've never heard it phrased that way. The only time I hear "the idea of" someone being said is when someone says something along the lines of, "you only love the idea of me," as in, you don't love the person for themselves, but rather, what you think they may be, or what you WANT them to be. So in these cases, I think it's similar, if we assume they're correctly written. "I dislike the idea of him" probably means something like, "I can't stand the thought of him," meaning the person saying this really hates whoever they're talking about. "I miss the idea of her," is the difficult one, I think. I'm not sure what is meant here. It doesn't really make a lot of sense, but I think the idea is that the person saying this just really misses whoever it is. In that case, though, it should just say, "I miss her." Otherwise, it sort of sounds like they never really knew her, but were really stuck on a sort of an idea they had of her. I hope this helps!

Zeyu, thanks for the excerpt from the script. The key is "He isn't one of us." That's the idea of him that Mary dislikes. He is from the "wrong" class, not an aristocrat. The "idea of him" also includes the fact that he will inherit Downton and the family fortune. Mary dislikes the whole idea of the title, the house, the estate and the money going to a person of the wrong class. It is a horrifying thought for her.