So, proper grammar for that sentence would be-
He does tell him that if he doesn't do that, he will kill him.
This sentence has a minor error in that because the subject is never stated, the pronouns he and him get confusing. That is also very vague.
The comma is necessary because the first part of that sentence is a dependent clause. A dependent, or subordinate, clause contains a subject and a verb or verb phrase but does not express a complete thought. As a result, it cannot stand alone as a sentence.
You use doesn't instead of don't because the subject (he) is singular. If the subject were plural (for example, they) you would say- they don't.
I hope that helps!