'On the morning of the third day' is the most elegant version. You could also say 'On the third day, in the morning' with the comma, it's necessary; or even 'On the third morning'. But your original version, though understandable, is not so clear or natural sounding.

You cannot say that because it is not clear which noun the adjective "third" modifies. You need to choose either "day" or "morning" to be the word modified by "third", but you cannot choose both. Notice how the other commenters have correctly chosen just one of the words. You could also say "On (or 'in') the third day's morning"

You're correct with your revised version. We'd likely say: 'On the morning of the third day' or 'On the third morning (of our trip)'

You can say 'on the morning of the third day' or ' on the third morning'