If you use "его", you'll get "I'm talking about HIS brother".
However, "свой\своя\своё" is a bit tricky. They mean "my\your\his\her\their" at once, depending on the person. Just compare:
Я говорю о своём брате. I'm talking about my brother.
Ты говоришь о своём брате. You're talking about your brother.
Он\она говорит о своём брате. He\she is talking about his\her brother.
Мы говорим о своём брате. We're talking about our brother.
Вы говорите о своём брате. You're talking about your brother. (plural or polite)
Они говорят о своём брате. They're talking about their brother.
But:
Я говорю о его брате. I'm talking about his brother.
Ты говоришь о его брате. You're talking about his brother.
And so on...