Understand it, don't use it. (Su.Ki. understands it better than I do!)
Here is "proof" that it is seen as pretentious. Normally, college football "fight" songs are full of lyrics like "We're gonna fight fight fight for every yard/And throw the other team upon the sod (Hit 'em hard!)/Scarsdale High!"
The satirical songwriter Tom Lehrer, however, wrote a Harvard fight song that portrays Harvard as a bunch of pretentious, stuck-up, effete, Anglophilic ("tea" and "how jolly!") snobs. Thanks to Su.Ki. I realize that "albeit" is being used incorrectly, and Lehrer probably did this intentionally. He uses a fancy word in place of a plain word wherever he can ("spheroid" instead of "ball.")
In his performance, https://www.youtube.com/watch?v=lWbU2DuWSGE , he gives a comic impression of the "Harvard accent" which still existed in the 1950s.
Fight fiercely, Hahvahd,
Fight, fight, fight,
Demonstrate to them our skill!
Albeit they possess the might,
Nonetheless we have the will!
How we shall celebrate our victory:
We shall invite the whole team up for tea!
Hurl that spheroid down the field, and
Fight, fight, fight!