Your written grammar, while not perfect, is fine, so I suspect you'll need to look for the problem elsewhere... perhaps pronunciation.

My grandfather came to Australia in his twenties without knowing a word of English. He saved money, bought a shop and learned English by interacting with customers as they bought stuff off him. Years later, he owns a business in Australia and rents out various properties, all of which he has to do in English. He watched either the BBC and the ABC, and he has no problems understanding anything. His grammar is far from perfect. Most of the time he completely ignores articles and tenses, uses the wrong prepositions and occasionally uses strange word order, but he has no problem making himself understood. He always has to have someone proofread anything he writes, and it's deciphering some of his writing can be surprisingly harder than understanding his speech. Nonetheless, I'd consider him a fluent speaker of English. Grammar is definitely important, but that doesn't mean it's the most important thing. I've met people who could speak English with perfect grammar, but they put sentences together slowly and struggled to understand what they heard. It's better to be like my grandfather than like them, but ideally you will be able to speak fluently AND with perfect grammar. Study grammar, but not to the exclusion of everything else.

Absolutely not. When I started my English lessons, I focused primarily on grammar and pronunciation. Later on, when I actually had conversations with native speakers, they told me that grammar, while being important, isn't the most important part to focus on. Majority of native English speakers speak so called "broken English" that is, English that doesn't exactly follow the grammatical patterns, but they understand each other, as well as other English speakers just as well.