I live in the Texas which is considered part of the South of the United States. Fly down usually means take an airplane to travel from north to south to visit. If I had a friend in New York and I lived in Texas, I might ask him to "fly down" and visit me. If you look at a flat paper map, north points toward the top of the paper or "upwards" and south points toward the bottom of the paper or "downwards." This is where the idea came from. Fly up, Fly down, Drive up, Drive down are all used and very often they are used incorrectly. I might say to someone who lives west of me, "drive over/drive up/drive down." Native English speakers often use these terms interchangeably so don't be confused if someone tells you to "fly down" even though you are not north of them. You could at this point correct them if you wanted to :)