"At First", "First of all", "In the first place" have the same meaning and can be used as a synonyme in a speech for example. (Means, The first thing I want to tell is...) But "first" has also the following meaning: -> The winner of a tournament is the "first" or the "best" athlet. -> Or if you are a supporter of an athlet you could say: "He will be first". "In first place" (instead of "in the first place) can also be used for somebody who announces the winner: -> "In first place we have X." I cannot think about other differences - but maybe some others could give some other examples/rules?