It's difficult to explain, because the expressions or words can also be confusing! We could say it means, "in the first place," but that does not mean "first place" as in #1 or best or a winner. Maybe these examples will help: "You don't like the way I'm picking up your sofa? Be glad I'm helping you at all!" "That play is so terrible, the actors should be happy they have any audience at all!" "I'm so angry at you, you're lucky I'm talking to you at all!"

Brad's described it well by relating it to "in the first place"(=to start with). You could also consider "at all" as similar to "even" - though they take different places in the sentence: "Consider yourself fortunate to have a man in the games at all." "Consider yourself fortunate to even have a man in the games."

The tones. I don't like it, not at all. = I don't like it, not even a bit. = I don't like it, not even at the very least.