We can use this to talk about emotions.
You are saying you won't stop feeling something.
I'll never let go of my love for my girlfriend.
I'll never let go of my anger for my girlfriend.
You could also use it in a more physical sense if you are holding something.
You can say
The dog won't let go of the ball.