"In the first place" is used in two ways :
to mean 'firstly', in a statement which is going to be followed by another one giving further support to a case or argument e.g. I didn't do the work, firstly [in the first place] because I didn't have the materials, and secondly, I wasn't able to get to the place.
The other use of 'In the first place' i.e. 'to begin with / to start with' is when referring to something that has happened, which may form part of an explanation or an enquiry. 'He should not have been there in the first place' : implies that if he had not been there, something negative [undesirable] may not have happened.
'Why was he there in the first place ?'
'If I had not done the work in the first place, this would not have happened.'
'For the first time' is not used in the same way.