Yes, my interpretation is the same as yours. There is a phrase--I don't know what to call it, it is less than a "saying" but more than a "colocation"--"us against the world."
A Google Books search turns this up an example in a book about preparing for marriage: "Since our wedding day we have held an 'us against the world' attitude in our marriage. We look out at the world and build prorelationship thoughts that foster emotional safety in our marriage and set the stage for everyday bliss."
I don't know if that's good advice but certainly couples in love often feel that they have created a circle of safety that protects them from an uncaring or hostile world.
In your passage, she feels that the danger from "the rest of them," from the rest of the world, is constant, and that if they let their guard down even for an instant--if a fight weakens their relationship--their relationship will be over. They are trying to keep out of the grip of "them," the rest of the world. She worries that "they," the rest of the world, will "have us," and he says "Don't worry, they won't get us."