Yeah... It's hard to explain when it's okay to omit words in casual speech. Of course, the sentence is wrong grammatically, but in conversation we often shorten sentences by leaving out words that aren't necessary. You've probably already seen plenty of conversations like this:
A: "You ready?" [Are you ready?]
B: "Not yet. Just a minute." [I'm not ready yet. I need another minute.]
A: "Is this the right answer?"
B: "Nope. Not even close." [No, it's not even close to being the right answer.]
The sentence you mentioned is just an example of this, although in my opinion it's a rather awkward example. In some cases, writers will leave out words that most people actually DON'T leave out in speech, because they think that leaving out more words will make their writing sound more natural or casual (or, in some cases, more dramatic). Some authors are great at this, but others aren't. It's a skill. It's very hard for me to imagine anyone in real life saying "No way for me to go back." To me, this sounds like the kind of omission that would be much more likely to happen in a book or a movie script than in an actual conversation.