What is the situation?

As I see it: I didn’t want to get you annoyed, I just wanted you to stop talking.

Maybe "I was just looking for the mute ...?" That's the button which turns off all the sound - the mute button, often called just the mute>

Pushing someone's buttons means to try to aggravate or annoy someone. As far as the last sentence I am not sure.