Either one is correct. The only difference is the focus in the sentence. "I am traveling by bus" specifies how you are going somewhere. You are going by the bus, not by car, train or airplane.
"I am traveling in a bus" is perhaps less likely to be used, but if you needed to talk about your exact current location, you could say this. If you are on the phone with someone, you might say "I'm in a bus now, so it's a little noisy."