I'm going to work at the office.

Both are correct, depending on your meaning. "At" refers to a place, so if you mean you will be working at the office instead of working at your home, then use "at." But if you mean you will be working in the office instead of working in the canteen downstairs, then use "in" to indicate the room you will be inside. You probably mean the first thing, so you should use "at."