Yes, it also means "to leave sb in peace" even in situations where no one is being bothered- for example: you are studying in a room and your friend who you live with is in the room with you. He is tidying up and he says "I'll just finish tidying the desk then I'll be out of your hair"