yes this is a common and natural saying in English. To be off something means to no longer be doing that thing. To be off salt means to no longer eat or use salt. He is off his diet means he is not longer following his diet. We often use it to mean that a person is no longer interested in something, specifically a hobby. For example, for years my mother enjoyed identifying the birds that would visit her garden. She read books, magazines, talked about it with us etc. Then one day, she's no longer talking about birds, something else has captured her attention. I might say to my brothers: mom is off the birds, now she's all about the butterflies.
`I hope that helps.