It is a somewhat archaic saying to do with handshaking etiquette. It means "forgive me/pardon me for not removing my glove," as it is/was expected for one to remove his or her glove when shaking hands with someone who is not also wearing gloves. The phrase is not very likely to come up today.

Have never heard that expression (where did you hear/see this?)...it not"'excuse me, my love" is it? :)