'just in case' means that you are putting the umbrella in the bag whether it rains or not. You are prepared for rain, but also fine if it doesn't rain. 'provided that' means that you will only put the umbrella in the bag in the specific circumstance that it IS raining. If it doesn't rain, you won't bring the umbrella with you at all.

In case

"in case it rains later."

in case