It means a lot of things. You should learn to post the context as well when you ask these questions. It also means the mind has gone blank. You are thinking nothing, just gazing into space.

Yes, basically, but "to gaze into space" usually means to look deeper into the sky, to see stars, planets, the Milky Way, etc., whereas "to look at the sky" is to look at things that are closer, like birds, planes and clouds.