1, 2 and 4 are all acceptable. "If my memory serves me well" is the full, standard British phrase though I believe Americans tend to use "right" instead of "well". It is a relatively sophisticated expression as it stands. Versions 1 and 2 are just abbreviations of this phrase and would be fine in conversation or in informal writing. I don't think many people would use version 3 - or, at least, I don't hear it. Another similar expression is: "If I remember correctly".

The last/longest one sounds most natural. And it already IS polite and sophisticated, more so than more straightforward expressions such as 'I can't remember exactly, but..'