It is not an easy question. I do not know about grammar, but I will answer with common sense.
If a verb is transitive and may have as an object the same who is speaking, it may be a reflexive verb. In this case, in Spanish you may add "se" at the end of the verb and it has a meaning:
When there could be doubt about the object on which the action of the verb falls, then, if you speak about yourself, you should use the reflexive, always with the gerund.
estoy estudiando: means I am studying... maths, literature, or whatever.
me estoy estudiando: means I am studying myself.
You can use both forms with some verbs: "estoy riendo" or "me estoy riendo"; "estoy paseando" and "me estoy paseando". In this last case, it is clear that you are speaking of yourself. But you could say "estoy paseando el perro".
I do not know if this is what you are asking, but when both forms are right, in my opinion, "me estoy" sounds more natural in most cases, but I speak for myself.
One more observation; with "estoy" you can use an adjective. For example, "estoy cansado" is right, but you do not say "me estoy cansado".
Naturally we, Spanish speaking people, speak without thinking of all these things. Simply it sounds right or wrong.
Hope it helps.