We need to be careful on the definition of "go." It means to depart slowly, but not to move slowly.

Hi To linger means to go slowly

The song is about a woman whose boyfriend is being unfaithful. Letting it "linger" means that their relationship is fading, slowly and painfully. She's still in love with him, and needs either to be loved by him, or have him removed from her life. Lingering often means staying longer than one is welcome.