In this phrase, "confidence" is related to the verb "to confide (in someone)"... that is, trust and share one's secrets with another. The person you can tell anything to is a confidant (or confidante, f). So when you take someone into your confidence, it means you are sharing secrets or private information with them.
Maybe this is not always a good idea, especially if you don't trust the person who wants to tell you something secret: "Don't take me into your confidence!"