These two sentences are both correct and commonly used. If you say the first sentence, I think you are emphasizing "when" he arrived the city. If you say the second sentence, I will expect you are going to say something else after it. For example「他昨天进的城，但现在不知道在哪」. The important part comes afterwards. And when I talk to my boss or someone elder, I will use the first sentence to speak in a formal way, while the second is more casual.

The two sentences are almost the same. About the first one"他是昨天进的城", you could consider "是" as "It was" instead of "to be", so the sentence will become"It was yesterday that he came to the city". And the second one"他昨天进的城" could be translated into "He came to the city yesterday".

基本没有差别。

可以这样说，他昨天进的城

他是昨天进的城，是强调句，强调时间是昨天。 他昨天进的城，泛泛而谈有这样一件事。