Normally, I would say that "longest journey" refers to length of time (stay), although often this would be phrased as "longest trip". It is a subtle difference, but "journey" usually means the act of traveling, while "trip" can include the entire event, such as a vacation. If you go somewhere on vacation, you could say "the trip was good" to describe the whole vacation, but "the journey was good" would normally just mean the travel part. To express distance, you could say "What's the farthest journey you have taken". "Longest" could mean time or distance, both are correct, but "farthest" only means distance in this case.