To me, the fear of making mistakes and feeling too shy to speak are the most challenging parts of the language learning process.
You can overcome these fears if you practice with people who help you relax and give you kind corrections. It is definitely possible. Just keep practicing.