Just have very simple sentences about what you want, for example "What time is the next bus?" or "Which way to the bakery?"
When you feel more confident with short conversations like this, then you can start with more complicated phrases.
I feel the same when trying to speak with Polish people, everything you want to say just disappears from your mind! This is completely normal, don't worry!
Best of luck
I always thought my English was very good until the day I scheduled my first trial lesson here on italki, lol! I was so frighten up that I could barely talk to the teacher, since all I could manage to do was stutter and hear my heart pounding really fast inside of my chest, hahaha, I just freaked out...
After a couple of classes I felt more comfortable to talk, and now it's way easier for me to have a conversation than it was in the beginning.
So my advice for you is to simply talk without the fear of making mistakes. We're not natives and your partner will know that, so all your mistakes will be forgiven, don't worry about that.
Hahaha, take care, and don't give up!