Great question! "Up" and "down" are used in many ways. When traveling, "up" generally means north and "down" means south. Australia is sometimes referred to as "down under" because of its location in the southern hemisphere. In cities, "downtown" refers to the center of the city and "uptown" refers to a less central part of the city. "Up" is used sometimes to mean "closer", as you have noticed. "Down" is not used in this way. Both words can be used in a more nonspecific way as synonyms for "there" or "over there". You could say "Yesterday I drove over to Mary's house" "Yesterday I drove down (or up) to Mary's house". I'd be more inclined to say "up" if Mary lived to the north or closer to the center of town. You can't get in trouble by using the wrong word, because there is no wrong or right word.

Yes. Sort of. It depends :-) At times it does literally refer to if a thing is physically up or down. And in those examples you cannot change 'up' for 'down' or vice versa For example: "You are on the rooftop? Okay I will come up and meet you." It's not possible to change 'up' for 'down'. But in many instances the 'up' is just part of a phrasal verb and has no relationship with any change in height. The journey can be completely flat with no change of altitude. You even hear people saying things like "Shall we walk up to the pub now, or have something to eat first and walk down there (to the pub) later?" Bizarre isn't it! ;-)