The first sentence ("I shouldn't have stopped my studying English") is grammatically correct (use a possessive pronoun with a gerund), but in everyday English, no one would say that. At least not in the U.S. We would just say either, "I shouldn't have stopped studying English," or "I shouldn't have stopped my study of English." I think this is for two reasons. First, the "my" is implied already because the subject of the sentence is I. Second, the possessive + gerund combination is becoming less and less common in everyday speech and writing. It was a rule when I was a kid, but that's been a while. :)
Your second sentence is correct. It could mean either "I should have started studying English as a child instead of as an adult," or "I should have started studying English this morning rather than this afternoon."
I hope that helps!