To me, these two phrases become more confusing when they are spoken rather than when they are written.
But I think this is all the brain doing. It would force its own logic and sense (however unreasonable they were) on the actual information from the environment instead of processing it as it is.
Here's an article which talks about this issue in an easy and clear way:
A very interesting topic, by the way!