These two sentences are both correct and commonly used.
If you say the first sentence, I think you are emphasizing "when" he arrived the city.
If you say the second sentence, I will expect you are going to say something else after it.
For example「他昨天进的城，但现在不知道在哪」. The important part comes afterwards.
And when I talk to my boss or someone elder, I will use the first sentence to speak in a formal way, while the second is more casual.