I think that the two expressions are very similar to each other .
but for me , "by my self " means that you study with no help from any outsider .
" on my own " : you study with the help of someone but you finance your studies with no help .
I would say that " by my self " is a more private and strict expression than " on my own ".