It's great that you started with listening and conversation: that will give you a great intuitive sense for how to use the language. Now, you have two choices: you can continue with listening and conversation, and slowly improve your grammar. This is how babies do it, and it works, but it takes a decade and a half, just like it does for natives. Or, you could read a grammar book, do lots of grammar exercises, and take lessons from a grammar-focused teacher. This definitely isn't a substitute for speaking: there are skills that you can only develop by actually using the language. However, reading a grammar book can give you a great top-down view of the language, and can give you insights that would take much longer to develop by yourself. I'd also recommend a lot of reading in English: reading lets you see grammatical patterns that are harder to notice when you are just listening to speech.

Study with good language materials and/or a good teacher. Practice, practice, practice!