Both are fine. You could also say He’s late by only a couple of minutes.

Soy gringo. Both versions are okay. Ambas versiones están bien. En esta situación, "sólo" funciona de manera similar a "simplemente" o "meramente" o quizás "solo". La idea es que la cantidad sea tan pequeña que no sea gran cosa. Por ejemplo: - The shoes only cost five dollars. - The store is only 10 minutes away. - I am only joking. - I have only known him for two days. ¡Buena suerte!

Both are fine. The first one is equivalent to "He is only late (by) a couple of minutes."

The 2nd sentence is correct, in the first one you should have said "he is only late by couple of minutes"

Both are fine, but I think the second one is slightly more common.