The problem is not the question - it's the answer. When we're telling someone what we do for a living starting with 'I'm a ...' we either give a profession:
I'm a teacher
I'm a dentist
or we give some kind of job title:
I'm a restaurant manager
I'm a production supervisor
The term 'office worker' is neither a profession nor a job title. I suspect that 'office worker' is a literal translation of a term in Korean, and doubtless in many other languages. And whereas speakers of other languages may well say 'I'm a [ equivalent of 'office worker' ], native English speakers don't often say this.
We might possibly say 'I work in an office', or more likely 'I work in a tax office' or 'I work in the office at a Costco supermarket', so as to give more information. Or we might say what you actually do, what position or role in the office actually is. For example, we might say 'I work as an admin assistant in an insurance company.'
To the native speaker, the term 'office worker' sounds a little odd - either foreign or old-fashioned.
I know many people who work in offices all day, and I can't imagine any of them chatting to someone at a party in the evening and saying 'I'm an office worker'. It's just not what we say.