I think you are trying to make it similar to when we say "He's a one-woman guy," meaning he only goes for one woman at a time. In that case I think it would be "He's a three-kid guy" would make sense, but it would mean he is the type to want or have only 3 kids and that's it. But to say he's the father to 3 children, I wouldn't say it.