I'll have a go at answering this question.
have a go at something = make an attempt, or try
You can use it in many ways, but one variation is "start a fight" - to have a go at beating someone.
So, if you get into an argument with a taxi driver, he may reply "oh yeah, do you want to have a go?"
Over time, the usage has changed, and it can mean to criticise or challenge someone. That's what your sentence means. You have a good reason so I shouldn't say anything.
BUT, it doesn't refer to your emotion. You can be angry, but choose to stay silent. You can also be quite calm, but still have a go at someone. For example, my afternoon class is very lazy because they just ate lunch and are tired. I need them to try and put more energy into the class, so I have a go at them - speak strongly - to let them know it's not OK to sleep in the class. (And then I make a joke to let them know I'm not really angry with them.)