My perspective is that "annoyance" or "annoy" focuses the person, thing, or action that is being annoying e.g. "HE was annoying me so much" (he is being annoying) whereas 'irritation" or "irritate" focuses on the person or thing that is being irritated e.g. He was irritating ME so much (I was feeling irritated). These are just my personal connotations of these words; "annoyance" is when you want to stress the external factor being annoying, and "irritation" is when you want to stress that you felt irritated. Also, irritation is used medically for skin irritation, bowel irritation etc (annoyance is not).

Essentially the both mean the same thing; to be annoyed is to be slightly angry. It is an emotional feeling that one has. You can be annoyed, and someone can be annoying to you. Irritated, however, can mean not only an emotional feeling, but it can also be physical. You can say “his skin was red and irritated”. It can involve an aspect of the body, as in “his eyes showed slight irritation”. Annoyed cannot be used in this way, it is strictly for emotion.