When someone says, "Get off my back" they typically want you to stop nagging them about something. It does not necessarily mean someone else is bothering them, too. This is not a kind expression to use. You could say it is rude, but usually the other person has provoked you by excessive nagging. To be more diplomatic, you could say, "I'd rather not talk about that." at first and if they are insisting on it you could then say "Get off my back."

Yes, it means "leave me alone!", but it doesn't have to mean that anyone else is bothering them. Usually, it's said if someone keeps nagging or repeatedly bothering someone about the same issue. While it's not vulgar, it can come across as a bit snappy, rude, or defensive. For example, if my annoying younger sister kept bugging me about my eating habits and wouldn't leave me alone, then I might get frustrated and tell her "Get off my back!" But if my sweet old grandmother was bugging me about the same thing, I might say something like "Thank you, but I'd rather not talk about it." ...exactly how Del explained it above!