If you want to improve the listening skill, you have to tune your ears to the language you are going to cope with. As the very first step you should start with some simple things: podcasts for beginners and movies like "Friends" with subtitles.
But the understanding of what you hear should be interlaced with your grammar knowledge. You should know why that's used this way and not that. Without grammar you can train your ears to distinguish the sounds, but not to understand the main idea of a sentence. Or it will take more time for that, like little kids do that. Develop all the skills simultaneously. The more words you know, the more you can distinguish ;) unconsciously