It's a not-very-tasteful joke about (Roman) Catholics having large families, which was often the case, as contraception was restricted or unobtainable in countries where the Catholic church was dominant. He could not have been a Catholic because the Catholic Church did not allow divorce, but he had a lot of children, which suggested that, on this criterion, he could have been.

the stereotype of Catholics disbelief in birth control.