I see this as a type 1 conditional in the past being reported with the usual verb backshifting.
At the moment:
He says: "Must we pursue this horrible trend of thought?"
I think: I am sort of afraid that he will get up and leave on me, if I don't shut up.
In the retelling:
"Must we pursue this horrible trend of thought?"
I didn't say anything. I was sort of afraid he'd [he would] get up and leave on me if I didn't
I am afraid => I was afraid
he will get up => he would get up
I don't shut up => I didn't shut up