Try to practice, while asking the other person to please correct you. If there is something you don't understand, always ask. I'm still not confident, either, but it certainly won't get better by not trying.
Ask yourself what are you afraid of?
Not being understood? That can be explained.
Making a mistake? It can be corrected.
Offending someone? They will understand it's language barriers.
Being teased? Then they are the mean one!