It means to leave or go away. It probably comes from "fade out" - disappear. "Fade" also means to cover a bet. You would hear it as "you're faded" = I've met your wager.

let's fade= let's go

When the scene fades in a movie or concert, then the sound and image disappear slowly They're just using movie language here.