Depends how well you want to speak.  If you just want to order food at a restaurant and get a couple of "wows" out of some people then no, you wouldn't need grammar for that.  But if you want to have a strong grasp of the language where educated people actually like having a conversation with you then you're going to have to learn the grammar.

Language learning is hard work.  It's not fun.  There is no immediate reward and in many cases no reward at all.  English being the only exception to this because speaking English unlocks almost the whole world for you depending on what you do for a living.

But apart from that it's pretty bleak.  Also, I recommend learning one language at a time.  You can always spot the people who have a hard time with mechanics because they learn how to say 20 phrases in one language, the "fun part" ends, and then they switch to another language.  So they'll have 1-2 pips in 5 languages but might as well not be any.

From my own experience I say that is better learning english or any other language without focusing too much in boring grammar rules. In fact, I used to study grammar everyday consistently and making a lot of boring exercises with boring and useless grammar text books, and I forgot the majority I learnt from them. They didn´t help me to have better writing, speaking and listening skills. I only lost my time and felt frustrated of no seeing any progress.  I agree with Soren words and since this year I started focusing on Input firstly and then output what I previously learnt. I think is better focusing on Audio input rather than Visual input (Reading) because no matter how you slice it language is essentially an Audio skill (even if you are reading in your native language, you will notice that you will speak with yourself) In youtube or Itunes you will find a lot of funny audio input and some of them come with its transcripts. I recommend you to listen many times the same materials until you are familiar with all the words, and you are fully capable to use those new words automatically and naturally. Don´t worry about mistakes because they are your friends the more mistakes you make the faster and better you will speak your target language. Grammar books also didn´t help you improve your mistakes, whereas input does. Find as many friendly language partners  as you can and don´t be afraid of go out your comfort zone. Another helpful advice is to mimic actors or actresses of tv shows that you enjoy. It is not only very helpful but also very funny. Good luck with your language learning. By the way, I share with you that passion of learning languages.

If you're talking about "speaking", as in conversing, I think it is possible as long as the person we are talking to understands what we are talking about.

However, if it's "learning", grammar has to be one of the learning curves in any language, isn't it? :)

I am learning Korean by watching Korean dramas and I can understand some Korean but I still need to use a grammar book to be able to speak it

It could be happen, but it's more better when you are speaking with knowledge of grammar.It's more likely a person who have an academic eduaction against who haven't.