This isn't a matter of grammar; it's a matter of style. And, as you've noticed, styles differ from place to place or publication to publication. According to the more popular style guides in the US, "alma mater" should not be italicized.

Technically it should be italicized as it is a Latin word and I suspect it should be capitalized as well however I'm not sure about the capitalization. If it's being used as a Title or Name then it will need to be capitalized. With that said, I rarely see non academic text italicize foreign words so I'm not sure it really matters. I've never bothered personally.