At their / your pleasure is not a normal thing to say. Is it possible you are confusing this with the phrase "" at your leisure ? "

Both seem very unnatural to me and I can't imagine using either of them.

This is not formal speech; rather something that is usually part of a dramatic monolgue in a bad movie: "Dont forget your place. You serve in this court at the pleasure of the king. Lose that, and you'll spend the rest of your life in the dungeon."

I agree with everyone who says that "at someone's pleasure" is an unnatural-sounding phrase. Don't use it.

You could. You probably shouldn't. "At their pleasure" isn't the kind of phrase that is used in every day speech. It only belongs in certain contexts and if you use it outside of those very specific places, it'll sound really weird.