Good question Camilo!
In my view:
That person is bothering me = they are a nuisance and slightly disturbing you, but you can cope with it. More like a pest.
That person is annoying me = they are actually making you angry, and you are slowly getting towards the point when you will lose your patience and act upon this anger.
That person is bugging me = the mildest of the three, often referring to a friend, or something that you can't remember. e.g. It was bugging me that I couldn't remember the word in Spanish.
I hope that this helps.
Also, "to bother" is an action. For example, "The beggar was walking down the street bothering all of the passers by".
Tom