French makes me anxious

I love French don't get me wrong. I watch movies,  listen to music, read books( recently readig Eleanor et park) but when it comes to speaking 😰😰😰 i panic.

That day, i was in a supermarket and a French old man stepped on my foot and he said pardonne moi.  All i needed is saying c'est pas grave.  But i panicked 😳😳 and said nothing. Just standing there. I left feeling horrible, it happens all the time. 

Help please.

How can i come over this???