without looking it up I think that this expression really started out being:
I hate to think
But they come down to the same thing really - that the idea of picturing the consequences of something would cause mental pain!
Since you can imagine yourself taking some kind of pre-emptive strike on the situation the expression:
Perish the thought
comes down to pretty much the same thing. Since I hate to think of it why not destroy it completely. Perish the thought has a bit of humour added to it since it is dramatically worked.
It doesn't bear thinking -- would be much the same as I hate to think.
Let's not even go there -- would be a very informal way of saying it, though this would be more like "I don't want to talk and think about it with you."
It really means, the idea of what could happen is such that I would rather not think about it.
Now, I realise that all my explanations need more English knowledge than the first statement.
It just means "I don't want to think about it"
Now why didn't I say that in the first place?!!