"I grant you that" is the same as saying "it's true" or "you're right". You are confirming a fact. Usually, there is a "but..." clause. It's a lot of risk, I grant you that, but it's worth it. It's a lot of risk, it's true, but it's worth it. I grant you that it's a lot of risk, but it's worth it. The watch is expensive, I grant you that, but it's beautiful. The watch is expensive, you're right, but it's beautiful. I grant you that the watch is expensive, but it's beautiful.

Yes, it sounds natural and you could also use "I agree with you."