You can say 他在部队呆过。 Other answers are good . My answer is short, so maybe it is easy for you to say . He - 他 Used to be - 呆过（our difference is that we translate these words in different way) In the army - 在部队

他以前在军队呆过 If he was a soldier, you can say 他以前是军人

他以前在部队。

This translation below is a bit more formal: 他以前在部队服役。( He served in the army.)

他以前在军队里