The sentence is grammatically correct, but not natural. It's unnatural because it's not clear what you mean. As Austin says, it could mean several things.
I may be wrong, but I think you meant, "We have so much things to talk about when we are alone together".