"Wearing your heart on your sleeve" is common English idiom. It means to reveal your emotions for all to see, and not hide your feelings. "When I'm deep inside of me" refers to when he is preferring to be alone with his problems, or contemplating deeply.

@romulus i thought you like "kiss" honestly billy joels is a cool dude.. cowboy...