In your example it’s neutral and doesn’t imply irritation. There’s no difference between ‘it’s ok’ and ‘that’s ok’ in tone. ‘That’s ok’ is more likely to be used when making an enquiry, or if there is doubt about something, while ‘it’s ok’ is appropriate in a context like the one in your example.

In your example, B is not being rude or dismissive. In general the tone could explain better and yes /It's okay/ can be used to push people off and sort of shut them up abruptly.

B is not angry

According to your example, you would rephrase “I forgot buying” with the infinitive of “to buy”. However, according to the example, it could potentially convey irritation.

It would be purely contextual and more conversationally implied. So if the speaker tersely interrupts you to interject “that’s ok” it could imply the speaker is being dismissive of your message.