It's hard to know what you mean exactly and more context would help. However, in my view, it's not likely to be a correct sentence. You would need plural "exceptions". Do you mean : "The three of us are the only exceptions within the whole group"?

Yes, this phrase is correct, however, it sounds very literary and formal and would never be spoken, only ever written.