If you set out for someplace, you start the trip to go there. If you set out for London, you start the trip from wherever you are to go to London.

set off on a journey = start a journey set off for Mecca set off for home set off for work

I think yes it is.