I agree. To my mind, if I say 'I was ready to go outside', it means I was ready and prepared, but not about to do so - I was probably sitting at the table waiting for my wife to hurry up.
But if I say 'I was set to go outside', I was literally about to open the door, and step out.
But that is a very tiny distinction, in practice you will be understood whichever you use.