I had about 300 lessons in Mandarin and had the same fears. Most of my lessons were with community tutors and we talked around the same subjects building up familiarity in commonly occurring topics.
Two things really helped me:
1) Increasing listening skills - IMHO, this is more than half the battle. You get much further if you have good listening skills with limited speaking skills than the other way round. Once you start to understand people in front of you, you have less inclination to run away.
2) massively increasing practising repetitions of sentences by listening and shadowing until I could speak fairly smoothly and practicing again. I was only doing less than 10 reps a sentence. It takes one to two hundred or more of purposeful reps to get a sentence out smoothly without hesitation. Couldn't believe the effect it had and I have been kicking myself for not doing it earlier.
These two study tactics will improve your confidence which is what is required.
Other tips: minimise reading when practicing speaking. You don't have a piece of paper with Japanese stuck to the front of someone's face when you talk to them.
Recently, I went to China and had no choice but to speak Mandarin. It went well. :-) Guess what study tactics I will use when I start learning Japanese... haha.
