They may or may not be interchangeable. Don't you love languages?
example of an interchangeable situation
Home alone: Home by himself
Examples where alone can't be replace with by ...self.
Oh my God, I felt so alone (lonely).
That's not enough burgers; the kids alone will eat that, never mind the adults (just the kids without the adults will eat all that and then there will be nothing left).
I alone know the secret of the tomb (only I).
And there will be others.
If they are interchangeable I would use either, or even "on their own".