The phrase "go away to somewhere," as in your example, means to travel from the current location or place to the specified destination. So, in your sentence, "My family went away to New York," it means they traveled from their current location to New York. It does not mean they went away from New York.

The "away" part simply implies it is a bit far away.