You must let them know. There is no need to feel bad to tell. From my experience, Japanese are usually too shy to express their negative feelings.

Just ask them to keep a lower profile. Tell them that you are still a beginner and you will not manage to understand their language at this level. Usually they should react by using simpler English,but some don't really care and go on as if you said nothing. That is many times due to them being inconsiderate or on the other hand they are not professional enough to adjust their level to suit their partner. In this case if they are not reactive to your warning , consider looking for another suitable partner ,being suitable after all is partly being able to suit your needs ,when learning a language.

Ask them immediately to stop and explain the expression. If you do this often enough, they will either stop using such difficult expressions, or they will stop talking to you and you can find another language partner. Either way, you're better off!

Just say very politely, "Hey, I'm just a beginner, can you slow down a bit and use simpler words?"

Yell at them.... :P