Tom: "Where are you going? The party has just started." John: "I'm off [going] to work now. Today I work a late shift because someone called in sick." The expression "off to work" reminds me of a well-known song from the classic children's movie "The Wizard of Oz": "We're off to see the wizard, the Wonderful Wizard of Oz. We hear he is a whiz of a wiz, if ever a wiz there was..."

Another point. 'I'm off to work' and 'I'm off work' are very different. The second example means you are sick and you are not currently going to work. This could refer to just a few days or possibly an extended period of time (months) maybe due to depression or a more serious illness. HtH Rob.

It means 'I am leaving now to go to work'

It means you are going to your place of employment/job - the word 'off' means the same as 'setting out' I'm Setting out to work/ I'm going to work