In colloquial English this is correct and it makes sense. Yes, technically, you don't have to use the "but," but in this context it's a simpler way of saying:
"Sorry for not answering. The reason that I did not answer is because I was at work that time until 11PM."
Using the "but" just makes the sentence easier and sound better.
Also, the "but" helps soften the sentence. It's almost used as a combination of an explanation and an apology.
Hope this helps. Good luck!