Salu Fredo
‘Just two of us’ = only two people from a group of more than two (including the speaker), but not necessarily referring to the speaker. Here’s an example:
‘I have a large family; we are all going to the lake, but only two of us will have a swim.’
If you add the word ‘THE’ to the phrase this will change the meaning:
‘Just the two of us’ = only the speaker and either the person they are speaking to or another person who is with the speaker.
Sometimes - You and I.
Sometimes - Myself and the other person I am with. Another example:
Fredo: “My friend and I would like to book a table for dinner this evening, please.”
Waiter: “Certainly, sir. For how many people?”
Fredo: “Just the two of us, thanks”
In this example, Fredo is not suggesting that the waiter joins him for dinner!