You have to learn grammar because it's the hardest part of this language. The second thing is talking and talking, without this you won't get basis to understand this language well. Try also to concenrate on vocabulary. pronounce isn't easy. You should watch also some polish movies and series. I don't know how often you learn but if you do this every day it won't take longer than about two years. Trening czyni mistrza, powodzenia! 

I understand your struggles. For me it took about 4 to 5 months to be able to speak freely without having to think too much. What helped me alot was the fact that my girlfriend is Polish and does not speak any other languages than Polish. Also at work I had a lot of Polish contacts. This way, speaking polish daily, will pave the way for you to become more confident. Also, I found that as a non Slavic language speaker it is quite usefull to put yourself to understanding the rules of grammar as quickly as possible. It takes a while, but eventually it will become hardwired into your brain.

If you need some Polish converstion I can help you :)
just let me know :)