If you are telling a story and referring to a morning that has already passed, you would say "On that morning..."

I think: in that morning.