I would say "He is first in Math" if you were talking about a class or course, however if you mean mathematics in general, I would say "I'm good at math".

Thank you, Gee

In British English, we say "maths" and in American English, they say "math" As for your question, we would most likely say, "My friend is top of the class in Maths" and not "My friend is first in Maths"