Correction 1. How to I say"sorry for my late reply" in Japanese?　※ How to I say→ How do I say 2. 本当にありがとうございます。→よろしくおねがいします。I hope you will take care of me.＝Thanking you in advance. To say ありがとう is strange because you haven't known whether someone gives you a favor. It might be a correct expression in English. But in Japanese, we don't say "ありがとうThank you" before we know someone gives us a favor. Answer "返事が遅れてしまい、申し訳ありません" is correct when you reply after passing the time limit for answering. "返事が遅くなり、申し訳ありません" is used when you reply late.

your translation is correct! "遅れてしまい、申し訳ありません"　is a formal expression. when you talk with friends, "返事が遅れてごめん"　also works.