"I can't quit you babe" is something someone says to their lover and means "I can't leave you" or "I can't break up with you." And, yes you can say "I can't quite my job," but that's only for your job :)

Означает, "Не могу бросить тебя детка!" . Это американская фраза.