I think a combination of the two approaches will be effective.
Free conversation classes are useful to help you to start talking in your target language, but after a while it gets boring.
Structured classes are more demanding and oblige you to listen and do more homework.
Why limit yourself to one teacher?
I don't know why there should be a choice between conversation and structure. A good teacher doesn't force you through textbook pages or a generic lesson about describing pictures or filling in words on totally disjointed and uninteresting sentences. I think a good teacher provides structure but is flexible enough not to make you jump through pre-designed hoops of no importance or interest to you. The goal is to get to the point where the student speaks more than the teacher, so any exercises should be designed to get to that stage with a good foundation. I don't believe in impersonal and strictly by the book teaching.
I think you have your answer: you found the first class more enjoyable, which is important feedback for something sustainable long-term. It's not about wanting something fun or easy. I think that if we're challenged appropriately, it's enjoyable, even - and perhaps, because! - if we have to work hard to understand. We don't find enjoyable supposedly "professional" approaches that aren't based on what we really need to know and find interesting, and where there's no personal touch.
I have already a big experience of taking lessons here, and I can give you some recommendations that work for me, but not a fact will work for you.
First of all, you don't must to stick with one teacher. Better to have a teacher for structural lessons and a tutor or few tutors for conversations.
Second, you can take several lessons from one teacher before you make decision. It is really hard to find good teacher for intermediate level, just because teaching style need be different from beginners.
Third, you can have and try here as many teachers as you want, don't be shy.
And last, you have English as native, you can easily find a language exchange partner who is a good teacher.