Yes. They mean practically the same thing in this context.

"Late" is a much better choice because nobody can give birth to a child at an old age. It simply isn't possible. "Old age" usually indicates people in their 70s and above. "Late", however, is a relative term. What it means depends on who says it. To a 20 year old, "late" might mean in her thirties.

Neither make much sense in isolation because when we think of ‘old’ or ‘late’ age, we think of ages at which giving birth isn’t possible. If it were possible for me to give birth in my 60s, I wouldn’t want to. (Hypothetical) I don’t want to be an older mother. (=Older than most mothers) I’d never have a kid in my 40s. (Some women do, but that’s too old for me)

Late is generally not used, but you could say “later in life” and this could almost be similar to the second statement.

✅ "I would never give birth to a child at a late age." → This sounds more natural. "Late age" is often used when talking about pregnancy and childbirth. 🚫 "I would never give birth to a child at an old age." → This sounds awkward because "old age" usually refers to much later stages of life (e.g., senior years). It’s not commonly used in discussions about childbirth. A more natural alternative: 👉 "I would never have a child at a late age." (More common in speech) 👉 "I would never give birth at a late age." (More concise)